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Figure  6.18:  Scatter  plot  illustrating  the  relationship  between  static  data  SNR  and  Fit.  The  3rd 
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Executive  Summary 

The  MetalMapper  is  an  advanced  time  domain  electromagnetic  (TEM)  system  for  application 
towards  the  detection  and  characterization  of  UXO.  The  antenna  configuration  includes  3 
orthogonal  transmitter  loops  and  7  tri-axial  receiver  loops.  The  system  can  be  deployed  in 
mapping  (or  detection  configuration)  wherein  it  acquires  data  along  profiles  while  the  antenna 
platform  is  in  motion.  However,  the  most  important  benefit  of  the  elaborate  antenna 
configuration  is  that  it  permits  us  to  characterize  a  buried  metallic  target  from  measurements  at  a 
single  spatial  point  located  (approximately)  above  the  target.  This  system  is  being 
commercialized  by  Geometries,  Inc  (San  Jose,  CA). 

In  2008,  the  MetalMapper  was  demonstrated  at  the  Standardized  UXO  Technology 
Demonstration  site  located  at  APG.  In  2009  and  2010,  the  MetalMapper  participated  in  live  site 
demonstrations  at  the  former  Camp  San  Luis  Obispo  (SLO),  and  the  former  Camp  Butner.  The 
performance  objectives  of  these  demonstrations  were  to  demonstrate  the  capabilities  of  the 
MetalMapper  when  operating  both  in  its  mapping  mode  (target  detection)  and  in  its  static  mode 
(cued  ID  target  identification). 

The  system  has  a  detection  “foot  print”  of  approximately  1  m  ,  similar  to  that  of  the  EM61.  In 
detection  mode  surveying,  the  MetalMapper  surveyed  along  parallel  profiles  with  a  nominal 
offset  of  0.75m.  We  acquired  data  at  an  average  survey  speed  of  less  than  0.5m/s  in  order  to 
maximize  data  quality.  At  those  speeds  our  average  survey  production  was  slightly  more  than  1 
acre/day.1  Our  detection  scores  show  that  at  these  speeds  we  can  detect  all  targets  down  to  a 
depth  of  1 1  times  the  target  diameter.  Static  mode  survey  productivity  is  very  much  a  function 
of  terrain,  navigation  software,  and  deployment  platform.  Over  the  course  of  our  three  full 
demonstrations  (APG,  SLO,  and  Camp  Butner),  the  productivity  of  the  MetalMapper  in  the  static 
mode  doubled  from  approximately  200  pts/day  to  over  400  pts/day  reflecting  improvements  both 
in  our  navigation  software  and  in  our  deployment  platform. 

Our  performance  improved  from  one  demonstration  to  the  next.  We  attribute  this  improvement 
not  to  improvements  in  overall  data  quality  but  to  improvements  in  our  ability  to  identify  and 
extract  the  important  discrimination  features  from  our  static  target  data  and  in  an  improved 
understanding  of  the  technology  of  decision  theory  and  pattern  recognition.  At  APG,  for 
example,  we  achieved  a  discrimination  score  at  our  operating  point  of  a  Probability  of  Detection 
(Pd)  of  approximately  90%  at  a  Probability  of  False  Positive  (Pfp)  of  approximately  10%  where 
the  low  Pd  primarily  represented  deep  targets  that  were  not  detected  by  the  MetalMapper.2  In 
contrast,  at  SLO  discrimination  scores  were  based  only  on  detected  targets.  At  SLO,  the 
Geometries’  discrimination  score  at  the  operating  point  was  Pd=98%  with  a  Pfp~5%.  Using  the 
same  data,  other  “Data  Processing  Demonstrators”  generated  similar  results  thus  showing  that 
discrimination  performance  is  not  tied  to  a  particular  discrimination  methodology.  At  Camp 
Butner,  our  discrimination  scores  were  good,  while  those  of  some  other  demonstrators  were 


1  Sample  rate  is  typically  10  Samples/sec  and  the  MetalMapper  survey  speed  can  be  easily 
doubled  or  tripled  with  a  corresponding  diminution  of  data  quality  and  spatial  sample  density. 

2  To  preserve  the  ground  truth,  the  Aberdeen  Test  Center  (ATC)  scores  provide  only  a  depth 
range.  Most  of  the  targets  not  detected  at  APG  lay  in  the  depth  range  “8x  to  12x”  the  object 
diameter. 
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nearly  perfect.  We  conclude  that  the  data  quality  gathered  by  the  instrument  is  very  high,  and 
sufficient  to  support  further  work  in  processing  of  the  targets  for  discrimination. 

The  estimated  cost  of  deploying  the  MetalMapper  based  on  the  demonstrations  we  have 
conducted  is  high,  2  or  3  times  what  it  would  cost  to  deploy  commercially  available  systems 
such  as  the  EM61.  These  estimated  costs  are  high  because  our  demonstrations  were  conducted 
by-and-large  with  high-cost  personnel  and  we  conducted  field  operations  to  maximize  data 
quality  rather  than  productivity.  We  project  that  the  cost  per  crew  day  will  drop  substantially 
when  the  MetalMapper  is  routinely  deployed  and  the  labor  costs  drop.  As  it  is  now,  the 
MetalMapper  costs  approximately  $1  OK/ha  when  used  in  the  detection  mode  and  $20/target 
when  deployed  for  cued  ID.  Detection  mode  productivity  is  about  0.5  ha/day;  however,  at 
normal  survey  speeds  (~lm/s),  the  detection  mode  productivity  would  be  lha/day.  Cued  ID 
mode  survey  productivity  is  300  targets/day. 

Higher  costs  and  lower  productivity  in  the  detection  mode  can  only  be  justified  when  we  can 
demonstrate  that  through  more  advanced  detection  methods  the  MetalMapper  can  eliminate 
many  superfluous  targets  such  as  small  surface  clutter.  So  far,  however,  we  have  not  developed 
or  applied  advanced  physics-based  target  detection  principles  and  therefore  we  cannot  justify  any 
claims  of  superior  target  detection  performance  relative  to  simpler  systems. 

The  demonstrated  advantage  of  deploying  the  MetalMapper  lies  with  its  ability  to  discriminate 
targets  and  thus  to  produce  a  prioritized  dig  list.  With  regulatory  acceptance,  the  use  of  a 
prioritized  dig  list  would  reduce  the  number  of  digs  by  50%  or  more  (depending  on  how 
conservative  the  dig  policy  was  set).  This  would  result  an  overall  cost  savings  of  30-40%  in  the 
cost  of  geophysical  surveying  plus  digging.  Therefore,  the  high  cost  for  the  deployment  of  the 
MetalMapper  is  justified  on  the  basis  that  it  can  substantially  reduce  the  cost  of  digging. 
However,  routine  deployment  of  the  MetalMapper  will  require  not  only  broad  regulatory 
acceptance  of  this  advanced  EMI  technology  but  also  that  it  or  other  advanced  systems  be 
specified  for  use  in  upcoming  UXO  remediation  projects. 

The  MetalMapper  is  now  a  commercially  available  product  from  Geometries,  Inc  (San  Jose, 
CA).  Geometries  is  able  to  provide  not  only  the  basic  hardware,  but  also  the  software  to  support 
the  data  interpretation,  as  well  as  training  and  field  support. 
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1.  INTRODUCTION 

1.1.  BACKGROUND 

There  are  millions  of  acres  of  land  located  at  formerly  used  defense  sites  (FUDS)  and  military 
installations  closed  under  the  base  realignment  and  closure  (BRAC)  program.  Much  of  this  land 
has  been  contaminated  with  unexploded  ordnance  (UXO).  These  lands  must  be  decontaminated 
and  restored  before  they  can  revert  to  civilian  use.  With  present  technology,  the  cost  of  the 
decontamination  is  estimated  to  be  on  the  order  of  10’s  of  billions  of  dollars.  The  high  cost  for 
decontaminating  these  sites  is  driven  by  the  need  to  visually  identify  all  metal  targets  detected 
with  current  metal  detecting  technology.  However,  only  a  few  percent  of  these  targets  are 
actually  UXO.  Because  of  the  potential  danger,  these  digs  must  be  conducted  by  explosive 
ordnance  demolition  (EOD)  technicians  resulting  in  digging  costs  in  excess  of  $  100/target.  At 
many  sites,  the  target  density  can  exceed  100  targets/acre.  Development  of  technology  that  can 
reliably  classify  metal  targets  at  a  cost  that  is  substantially  less  than  the  cost  to  dig  can  therefore 
substantially  reduce  the  overall  cost  for  decontaminating  these  sites  by  reducing  the  number  of 
required  digs  [2], 

The  MetalMapper  is  an  advanced  electromagnetic  induction  system  (EMI)  configured  for  the 
detection  and  characterization  of  UXO.  This  technology  represents  a  significant  departure  from 
existing  commercially  available  EMI  technology  (e.g.,  the  Geonics  EM-61).  Although  the 
MetalMapper  can  be  used  in  the  survey  or  mapping  mode  for  target  detection,  its  most  important 
application  is  for  use  as  a  system  for  “Cued-ID”  target  characterization.  In  the  cued  ID  mode, 
data  are  acquired  at  one  or  more  locations  in  close  proximity  to  the  target.  Precision  static 
measurements  at  these  locations  permit  the  calculation  of  the  target  characteristics.  For  isolated 
targets,  a  measurement  at  a  single  field  point  suffices.  These  target  characteristics  may  be  used 
to  generate  a  prioritized  dig  list  that  can  be  used  to  identify  targets  that  can  with  high  confidence 
be  left  in  the  ground  because  they  are  not  ordnance. 

1.2.  OBJECTIVE  OF  THE  DEMONSTRATIONS 

The  over-arching  objective  of  the  demonstrations  described  in  this  report  is  to  gain  acceptance  of 
MetalMapper  system  by  the  UXO  community  -  including  regulators.  To  achieve  this  objective, 
we  must  show  by  participating  in  blind  demonstrations  that  the  MetalMapper  can  indeed  be  used 
to  “reliably”  prioritize  target  lists  for  digging  and  thus  reduce  the  number  of  false  positive  (i.e., 
non  ordnance)  items  that  need  to  be  identified  through  excavation. 

1.3.  REGULATORY  DRIVERS 

There  are  no  regulatory  barriers  to  applying  the  technology  embodied  in  advanced  EMI  systems 
such  as  the  MetalMapper.  Such  systems  represent  an  evolution  of  already  well  established  and 
accepted  EMI  systems  such  as  the  Geonics  EM-61  that  are  being  routinely  applied  for  target 
detection  in  UXO.  The  real  problem  is  to  convince  the  regulators  of  the  reliability  of  the 
discrimination  information  that  the  MetalMapper  can  provide.  The  ESTCP  program  office  (PO) 
is  addressing  this  issue  by  conducting  a  series  of  blind  live-site  tests  in  which  promising  new 
technologies  such  as  the  MetalMapper  are  invited  to  participate.  The  demonstrations  at  Camp 
San  Luis  Obispo  (SLO)  and  Camp  Butner  were  two  of  these  tests.  To  gain  regulator  acceptance, 
the  PO  has  involved  regulators  in  the  site  selection  and  other  aspects  of  the  test  design. 


ESTCP  Final  Report  MM-0603: 


3 


MetalMapper 


2.  TECHNOLOGY 


The  MetalMapper  was  employed  in  full  demonstrations  at  the  Standardized  UXO  Technology 
Demonstration  site  at  Aberdeen  Proving  Ground  (APG),  at  the  former  Camp  San  Luis  Obispo 
(SLO),  and  most  recently  at  the  former  Camp  Butner.  A  description  is  given  in  this  section. 
Further  details  on  this  technology  are  available  from  reports  pertaining  to  the  AOL  projects  that 
we  cite  in  section  2.2. 

2.1.  TECHNOLOGY  DESCRIPTION 

Geometries  is  commercializing  an  advanced  electromagnetic  induction  (EMI)  system  for  UXO 
detection  and  characterization.  Called  the  MetalMapper  (MM),  the  new  system  draws  elements 
of  its  design  from  advanced  systems  developed  by  G&G  Sciences,  Inc  (supported  by  NAVSEA, 
SERDP,  and  ESTCP),  and  by  the  Lawrence  Berkeley  Laboratories  (LBL)  with  support  from 
SERDP  and  ESTCP.  A  block  diagram  of  the  hardware  system  is  shown  in  Figure  2.1 


Figure  2.1:  Functional  block  diagram  of  the  MetalMapper  of  the  showing  the  major 
components  of  the  overall  system. 

2.1.1.  MetalMapper  Hardware 

The  MetalMapper  system  is  unique  and  innovative  in  several  respects: 
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1.  Multiple  Transmitter  Loops3:  The  MM  antenna  platform  includes  3  mutually 
orthogonal  transmitter  loops  (see  photo  in  Figure  2.2). 

2.  3-Axis  Sensor  Array4:  The  MM  antenna  platform  includes  a  spatial  array  of  7  3-axis 
receiver  antennas  (21  independent  measurements  of  the  transient  secondary  magnetic 
field). 

3.  Electronically  Switched  TEM  Transmitter  Loop  Driver:  The  MM  system  is  unique  in 
its  ability  to  drive  its  transmitter  loop  array.  Under  control  of  the  DAQ  computer,  the 
output  of  the  transmitter  can  be  directed  to  any  single  loop  or  automatically  multiplexed 
between  loops.  There  is  also  control  of  the  fundamental  waveform  period,  duty-cycle, 
and  pulse  polarity.  Typically,  however,  the  loops  are  driven  with  a  classical  bipolar  pulse 
type  TEM  waveform  (i.e.,  alternating  pulse  polarity  with  a  50%  duty-cycle).  Depending 
on  the  survey  mode  (e.g.,  Static/Dynamic),  the  fundamental  frequency  of  transmission 
can  be  varied  over  the  range  1 . 1  l<f  <8 1 0  Hz. 

The  photo  of  the  MetalMapper  shown  in  Figure  2.2  was  taken  during  the  demonstration  that  we 
conducted  at  the  standardized  UXO  test  site  located  at  Aberdeen  Proving  Grounds  (APG). 
During  that  demonstration,  we  towed  the  MetalMapper  with  a  garden  tractor  in  order  to  acquire 
dynamic  data  at  a  uniform  survey  speed.  The  system  is  normally  deployed  on  skids  to  minimize 
platform  movement  over  undulating  terrain.  The  antenna  platform  including  skids  and  optional 
wheels  weighs  approximately  160  lbs  (72  Kg)  and  can  be  moved  by  hand  when  necessary.  At 
APG,  cued  ID  (static)  data  were  also  acquired  with  the  antenna  platform  deployed  as  a  towed 
system. 

We  envision  the  MM  Production  system  to  be  a  system  that  has  a  base  configuration  with  two 
primary  options.  We  will  call  option  1  a  Mapping  system,  and  option  2  a  C ued-ID  system.  Both 
systems  will  employ  the  same  data  acquisition  system  although  it  will  be  tailored  as  needed.  The 
Mapping  system  will  be  a  subset  of  the  Cued-ID  system.  It  will  consist  of  a  single  transmitting 
loop  identical  to  the  Z  (horizontal)  transmitter  loop  in  the  Cued-ID  system.  Moreover,  the 
mapping  system  will  employ  the  same  tri-axial  cube  sensors  but  there  may  be  fewer  of  them. 
The  mapping  system  is  intended  to  provide  maximum  ease  of  use  in  the  field  with  some  cost 
savings  at  purchase. 

The  prototype  MetalMapper  antenna  platform  is  pictured  in  Figure  2.2.  It  has  three  Tx  loops  in 
the  Z,  Y,  and  X  directions  and  it  contains  seven  tri-axial  sensors  inside  the  Z  (bottom)  loop.  We 
plan  to  offer  it  with  optional  hard/soft/large/small  wheels  and  it  will  also  be  available  with  a 
skid/sled.  The  handle  is  adjustable  in  length  and  height  to  allow  for  operator  preferences  and  for 
either  machine-powered  or  man-powered  surveys. 

The  data  acquisition  system  (DAQ)  is  built  around  a  commercially  available  product  from 
National  Instruments,  NI.  The  DAQ,  EM  transmitter,  and  batteries  for  the  entire  system  are 
packaged  in  an  aluminum  case  that  weighs  approximately  43  lbs  when  all  four  batteries  are 


3  The  3  transmitter  loops  is  a  feature  the  MM  has  in  common  with  the  AOL2,  LBL  BUD,  and  the 
USGS  ALLTEM  systems. 

4  The  MM  utilizes  3-axis  receiver  antennas,  developed  by  G&G  Sciences  for  the  Navy’s  AOL 
system,  to  acquire  measurements  of  the  vector  magnetic  field.  The  antennas  are  small  and 
approximate  an  observation  of  the  field  at  a  point. 
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installed.  For  surveying,  the  aluminum  case  can  be  mounted  on  a  pack  frame,  on  a  separate  cart 
such  as  a  hand  truck,  or  on  a  vehicle  such  as  an  ATV  or  lawn  tractor  (see  Figure  2-2).  At  SLO 
we  mounted  the  MetalMapper  in  a  skid  connected  to  the  front-end  loader  of  a  compact  Kabota 
tractor.  This  deployment  method  improved  our  productivity  for  static  data  acquisition  since  it 
was  easier  to  precisely  locate  the  MM  antenna  over  a  static  target  site. 

In  addition  to  the  Panasonic  terminal  shown  in  Figure  2.1  A,  the  instrumentation  package 
includes  two  external  modules  that  provide  RTK  GPS  and  platform  attitude  (i.e.,  magnetic 
heading,  pitch,  and  roll).  These  modules  are  connected  to  the  DAQ  through  serial  RS232C  ports. 
The  NI  data  acquisition  system  is  a  full-featured  PC  running  Windows  XP.  It  contains  disk 
storage,  serial  and  USB  input/output  ports,  and  more.  It  is  interfaced  to  analog-to-digital 
converters  and  to  digital  input/output  devices  through  its  internal  PCI  bus.  It  is  packaged  in  an 
industry  standard  PXI  configuration  that  is  intended  for  industrial  applications.  It  contains  a 
wireless  link  that  allows  the  operator  console  to  be  remote  from  the  DAQ.  The  Panasonic 
Toughbook  touch-screen  terminal  is  used  as  a  dumb  wireless  terminal  that  does  no  processing. 
Figure  2.1  B  is  a  functional  block  diagram  of  the  DAQ  instrument  package.  The  DAQ  is  also 
interfaced  with  a  GPS  unit  and  a  precision  attitude  module  through  two  RS232  serial  ports. 


Figure  2.2:  Photos  showing  the  MetalMapper  antenna  array  together 
with  the  electronics  package  and  control  console  (inset).  The  photo 
was  taken  at  a  demonstration  conducted  at  Aberdeen  Proving  Grounds 
(APG)  in  September,  2008. 
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2.2.  TECHNOLOGY  DEVELOPMENT 


May  04  May  05  May  07  Oct  08 

Demo-BP  Demo-BP/IH  YPG  Demo  Demo-APG 


1 - 1 - 1 - 1 - Tj^-| - 1 - 1 — ^1^  - 1 - 1 - 1 - 1 1^1 - 1 - 1 - 1 — I 


Jan  03  AOL  Project  jun  05  Jan  06  AOL2  Project  Dec  08 

(NAVSEA/Blackhawk)  (NAVSEA/G&G) 


Oct  08  Jun  09 
Demo-APG  SLO 


1  .  J 

1 1  1  1  1  1 - 1—1 - 1 - 1 — ^ 

r ' 

Jun  06  Metal  Mapper  Project  Dec09 

(ESTCP/Geometrics) 


Jan  04  Jan  05  Jan  06  Jan  07  Jan  08  Jan  09 

Jan  2003  Dec  2009 

Figure  2.3:  Time  lines  for  the  MetalMapper  and  related  projects. 


The  MetalMapper  uses  3rd  generation  technology  whose  development  began  in  2003  with 
support  from  NAVEODTECHDIV  (Indian  Head,  MD).  A  follow-on  project,  awarded  to  G&G 
Sciences  (Grand  Junction,  CO)  and  supported  by  NAVEODTECHDIV,  continued  hardware 
development  but  was  primarily  aimed  at  the  perfection  of  field  procedures  and  methodology  to 
make  deployment  practical.  And  finally,  with  support  from  ESTCP,  Geometries  (San  Jose,  CA) 
undertook  to  develop  the  3  rd  generation  hardware  and  conduct  more  comprehensive 
demonstration  in  anticipation  of  commercializing  the  technology.  Figure  2.3  shows  the  time¬ 
lines  for  each  of  the  3  projects.  In  the  following  sections,  we  summarize  the  results  of  the  two 
AOL  projects.  The  system  described  in  section  2.1  has  the  same  capabilities  as  the  AOL2 
system  together  with  an  improved  and  much  more  portable  electronics  package,  improved 
acquisition  software,  and  an  improved  antenna  platform. 

2.2.1.  AOL  System  (2003-2005) 

The  AOL  project  (NAVSEA  Contract  No.  N00174-03-C-0006)  began  in  2003  with  Blackhawk 
Geosciences,  Inc  (Golden,  CO)  as  the  lead  contractor  with  G&G  Sciences,  Inc  and  Geometries  as 
sub-contractors.  The  first  AOL  system,  pictured  in  Figure  2.4  (upper  left),  was  assembled  as  a 
“proof-of-concept”  system.  It  was  “uneasily  portable”.  It  is  significant,  however,  that  the 
designs  for  the  tri-axial  receiver  cube  and  its  preamplifiers,  the  transmitter,  and  the  data 
acquisition  system  and  software  were  completed  under  this  project.  In  the  follow-on  (AOL2) 
project  and  in  the  MetalMapper  project  many  incremental  improvements  were  implemented. 
However,  the  MetalMapper  is  based  on  the  same  fundamental  design  and  technology  elements  as 
its  2  predecessors.  Details  on  the  original  AOL  system  can  be  found  in  the  two  demonstration 
reports  submitted  by  Blackhawk  Geosciences,  Inc  (now  Zapata/Blackhawk  Division)  during  the 
original  AOL  project  [3,4], 
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2.2.2.  AOL2  System 


Figure  2.4:.  Photos  that  illustrate  the  evolution  of  the  MetalMapper  from  the  AOL(l) 
system  deployed  at  Blossom  Pt  (2004)  to  the  MetalMapper  deployed  at  San  Luis  Obispo 
and  Camp  Butner. 


The  contract  for  the  AOL2  project  was  awarded  to  G&G  Sciences  (Grand  Junction,  CO).  The 
objective  of  the  project  was  to  develop  procedures  and  methodologies  for  practical  deployment 
of  the  AOL  technology  by  conducting  demonstrations  at  YPG,  Indian  Head,  and  APG. 
However,  the  achievement  of  those  objectives  necessitated  the  development  of  a  2nd  generation 
antenna  platform,  and  the  development  of  a  new  transmitter  circuit  design  that  permits  the 
selective  energizing  of  3  different  transmitter  loops.  The  AOL2  system  used  the  same  data 
acquisition  hardware  that  was  used  with  its  predecessor.  Along  with  the  obvious  changes  in  the 
hardware,  the  AOL2  project  provided  an  opportunity  to  overhaul  and  improve  the  data 
acquisition  software.  The  resulting  “AOL2”  system  is  pictured  in  Figure  2.4  (upper  right)  during 
its  deployment  at  the  YPG  test  range.  The  AOL2  project  included  3  demonstrations  conducted, 
respectively,  at  YPG,  Indian  Head,  and  APG.  The  demonstrations  at  YPG  and  APG  were 
conducted  at  the  Standardized  UXO  Technology  Demonstration  Sites  and  therefore  were  scored 
by  the  Aberdeen  Test  Center  (ATC)  [5,  6]  .  Reports  describing  the  activities  and  results  from  the 
3  demonstrations  conducted  with  the  AOL2  system  contain  details  about  the  system  [7,  8,  9], 
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2.2.3.  Other  Advanced  EM  Systems 

There  are  two  other  advanced  EM  systems  that 
incorporate  some  or  all  of  the  technology  developed 
as  a  result  of  the  two  AOL  projects: 

1.  Man-Portable  Vector  (MPV)  TEM  System 
(CRREL) 

2.  TEMTADS  (NRL)[1] 

Each  of  these  systems  has  their  own  unique  antenna 
array.  However,  the  respective  instrumentation 
packages  and  sensors  were  assembled  by  G&G 
Sciences,  Inc  and,  at  a  fundamental  level,  are  based 
on  common  technology. 


2.3.  ADVANTAGES  AND  LIMITATIONS  OF 
THE  TECHNOLOGY 

The  principal  advantages  of  the  MetalMapper  system  Figure  2.5:  Figure  contrasting  the 
over  existing  commercially  available  EMI  system  are  difference  between  single  coil  and 
two-fold:  multi-axis  coil  transmitter 

illumination!  11. 

1.  Multi-Transmitter  Target  Illumination: 

Because  the  MetalMapper  has  3  orthogonal 
transmitter  loops,  it  is  able  to  illuminate  or  stimulate 

a  target  with  primary  fields  from  3  independent  directions  from  a  single  spatial  field 
point.  In  contrast,  single  transmitter  coil  systems  require  the  transmitter  (and  receivers) 
to  be  relocated  to  other  field  points  so  that  the  primary  field  will  stimulate  the  target  from 
a  different  direction.  The  concept  is  illustrated  in  Figure  2.5. 

2.  Multiple  Multi-Axis  Receivers:  The  MetalMapper  employs  multiple  receivers.  The 
MetalMapper  receiver  array  consists  of  7  tri-axial  receiver  cubes  measuring  10cm  on  a 
side.  With  this  array,  the  MetalMapper  is  able  to  sample  the  transient  secondary  vector 
field  at  7  unique  spatial  points.  The  positions  of  the  receiver  array  are  provided  in  Figure 
2.6.  The  receivers  are  positioned  so  that  they  traverse  profiles  13-cm  apart  along  a 
survey  line. 


ESTCP  Final  Report  MM-0603: 


9 


MetalMapper 


The  MetalMapper  shares  two  fundamental 
characteristics  with  other  advanced  systems:  LBL 
BUD,  USGS  ALLTEM,  and  NRL  TEMTADS. 

All  these  systems  use  receiver  antenna  arrays  and 
transmit  multiple  fields  from  a  fixed  position  to 
energize  the  target  in  different  directions.  The 
other  systems  have  important  similarities  with  the 
MetalMapper,  yet  they  also  have  important 
differences.  These  differences  serve  to  distinguish 
one  from  another. 

In  Table  4,  we  have  tabulated  a  number  of 
important  characteristics  of  4  advanced  EMI 
systems  and  highlighted  characteristics  where 
there  are  important  differences.  These  systems 
differ  from  the  MetalMapper  as  follows: 

1.  Transmitter  Waveform:  The 

MetalMapper  and  TEMTADS  use  a 
conventional  50  percent  duty-cycle  bipolar 
waveform  (see  Figure  2.7).  By  contrast, 
the  BUD  system  employs  a  half-sine 
bipolar  transmitter  waveform  with  a  duty- 
cycle  of  about  25%.  The  ALLTEM  system 
employs  a  triangular  waveform.  The 
advantage  of  the  conventional  waveform  is 
that  it  approximates  the  measurement  of 
the  impulse  response  of  the  target.  The 
ALLTEM  (triangular  current  waveform), 
approximates  measurement  of  the  step 
response  of  the  target.  And  BUD’s  half¬ 
sine  transmitter  waveform,  a  pulse  width  on  the  order  of  350  ps,  approximates  a  response 
to  equal  and  opposite  sign  impulses  separated  by  about  350  ps.  It  is  significant  that  the 
BUD  system  is  operated  at  a  base  frequency  of  270Hz  and  the  resulting  transient  is  only 
measured  out  to  approximately  1300  ps  after  the  turn-off  of  the  350  ps  transmitter  pulse. 
Both  the  BUD  and  ALLTEM  systems  operate  at  a  fixed  base  frequency  (270Hz,  and  90 
Hz,  respectively).  Unlike  the  MetalMapper  and  TEMTADS,  which  can  be  operated  any 
one  of  a  range  of  user-selectable  base  frequencies,  the  BUD  system  requires  hardware 
modifications  to  be  operated  at  a  different  frequency. 
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Figure  2.6:  MetalMapper  receiver  locations. 
All  7  receivers  are  in  the  plane  of  the 
horizontal  transmitter  (Z)  loop. 


Figure  2.7:  An  MM  acquisition  block 
illustrating  a  bipolar  transmitter 
waveform  with  50%  duty  cycle  and  a 
repeat  factor  (N)  of  3.  Block  periods 
are  operator  selectable  from  33.333ms 
to  0.9  s  in  multiplicative  steps  of  3. 


ESTCP  Final  Report  MM-0603. 


10 


MetalMapper 


Table  2.1:  Table  highlighting  differences  between  the  MetalMapper  (MM)  and  two  competing 
advanced  EMI  metal  detection  technologies. 


MM 

BUD 

AIITEM 

TEMTADS 

Multi-Axis 

3 

3 

3 

25 

Waveform 

Conventional 

Half-sine 

Triangle 

Conventional 

Programmable 

Yes 

No 

Yes 

Yes 

Moment 

~120 

600 

504 

24 

X 

t- 

Base  Frequency 

Variable 

(30,90,270) 

270 

90 

Variable 

(30,90,270) 

Receiver  Loops 

21 

16 

10 

25 

Multi-Axis  Rx 

7 

0 

0 

0 

X 

0£ 

Differential  Rx 

0 

8 

7 

0 

2.  Transmitted  field  geometry:  The  MetalMapper,  ALLTEM,  and  BUD  use  three 
transmitting  loops5:  a  Z-axis  located  at  ground  level,  and  two  horizontal  axis  loops  just 
above.  For  a  target  exactly  beneath  the  center  of  the  array,  the  three  transmitted  B  fields 
are  perfectly  orthogonal.  TEMTADS  uses  an  array  of  25  transmitter  loops,  each  35  cm 
square,  spaced  on  a  40  cm  grid.  The  target  is  illuminated  in  different  directions 
depending  on  the  geometry  between  a  particular  transmitting  loop  and  the  target.  If  nine 
or  sixteen  transmitting  loops  are  sufficiently  close  to  the  target  to  cause  a  measurable 
target  response,  the  target  will  be  illuminated  in  nine  or  sixteen  independent  directions. 

3.  Receivers  -  All  systems  employ  multiple  dB/dt  receiver  loops.  The  difference  in  the 
system  is  in  the  receiver  organization.  The  MetalMapper  uses  seven  3-component 
receivers  that  measure  an  approximation  of  the  vector  dB/dt  at  each  of  the  seven  points. 
By  contrast,  the  ALLTEM  and  BUD  systems  use  pairs  of  receiver  loops  that  are 
geometrically  null-coupled  to  the  primary  field.  ALLTEM  requires  this  null  coupling 
fundamentally  -  it  is  a  ‘primary-subtraction’  method.  BUD  implemented  this  null 
coupling  to  improve  receiver  performance.  One  advantage  of  using  receiver  loop  pairs  is 
that  they  can  reduce  common-mode  noise  (i.e.  uniform  B-field  noise).  A  disadvantage  is 
that  these  receiver  pairs  sense  a  finite  difference  field  that  usually  is  reduced  in 
amplitude.  TEMTADS  uses  twenty-five  receiving  loops,  each  25  cm  square,  oriented  in 
the  vertical  (Z)  direction.  Using  small  tri-axial  receivers  arrayed  in  a  horizontal  plane 
near  the  ground,  as  in  the  MetalMapper,  maximizes  the  signal  and  provides  horizontal 
field  components.  These  components  are  very  useful  for  near-surface  (<  0.5m)  and 
medium  (  0.5  <  z  <  1.0m)  depth  targets  and  often  have  larger  field  magnitudes  than  the 
vertical  component.  Further,  they  sense  field  direction  that  can  be  used  to  assist  the  user 
in  field  operations.  They  are  less  useful,  giving  a  smaller  signal,  for  targets  deeper  than 
lm,  but  none  of  the  systems  reliably  detect  smaller  targets  deeper  than  lm. 


5  Although  both  BUD  and  ALLTEM,  in  some  cases,  use  a  balanced  pair  of  transmitting  loops, 
the  effect  is  the  same  as  discussed  here  so  an  imperfect  comparison  is  made  for  the  purpose  of  a 
general  description. 
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The  MetalMapper,  BUD,  and  ALLTEM  have  been  designed  to  detect  and  characterize  targets 
within  a  radius  of  approximately  lm  from  the  center  of  the  horizontal  (vertical  axis)  transmitter. 
The  TEMTADS  array  is  2m  on  a  side  overall.  Experiments  we  have  conducted  together  with  our 
experience  in  the  field  with  the  system  demonstrate  that  the  MetalMapper  will  sometimes  detect 
targets  located  at  lateral  distances  up  to  2m.  Therefore,  in  areas  of  high  target  density  we  can 
predict  that  the  performance  in  the  discrimination  stage  will  degrade  due  to  the  presence  of 
interfering  anomalies.  This  is  a  likely  explanation  for  the  degradation  in  MetalMapper 
performance  between  the  Blind  Grid  and  the  Indirect/Direct  Fire  areas  at  APG.  In  the  case  of 
both  fire  areas,  the  target  density  was  often  quite  high  and  frequently  anomalies  were  closer  than 
2m. 

In  both  the  YPG  and  APG  demonstrations  we  have  deployed  the  MetalMapper  as  a  vehicular- 
towed  system  with  the  tow-vehicle  being  a  small  tractor.  However,  the  system  was  designed  so 
that  it  can  also  be  deployed  as  a  human-powered  cart.  As  a  human-powered  cart  the 
MetalMapper  has  limitations  similar  to  but  perhaps  more  severe  (due  to  greater  weight)  than 
those  that  apply  to  more  conventional  cart  systems  such  as  the  EM61  Mkll.  At  SLO,  we  again 
deployed  the  MetalMapper  as  on  the  front  end  loader  of  a  tractor  for  both  dynamic  mode  and 
static  mode  surveys.  Our  experience  at  Aberdeen  taught  us  that  a  vehicular  powered  system  has 
significant  advantages  compared  with  human-powered  systems  any  time  that  the  terrain  permits 
the  use  of  a  suitable  tow  vehicle.  At  SLO,  we  used  a  better  tractor  in  order  to  improve 
maneuverability.  That  tractor  had  four  wheel  drive  to  handle  difficult  terrain  and  it  has  the 
ability  to  lift  the  cart  to  allow  short-radius  turns.  Its  front  end  loader  allowed  for  rapid 
positioning  of  the  array  over  the  target. 
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3.  PERFORMANCE  OBJECTIVES 

We  have  conducted  three  full  demonstrations  (APG,  SLO,  Camp  Butner).  The  objectives  we 
outlined  and  explained  in  the  APG  demonstration  plan  are  basically  a  subset  of  those  we  have 
listed  here  from  our  SLO  demonstration  plan.  To  avoid  duplication,  therefore,  we  include  here 
only  the  objectives  from  the  SLO  demonstration  plan.  In  section  7  of  this  report,  we  will  use 
results  from  the  APG  demonstration  where  appropriate  to  demonstrate  that  we  have  been  able  to 
meet  our  performance  objectives  for  both  demonstrations.  Those  performance  results  from  APG 
also  serve  as  a  sort  of  benchmark  against  which  we  can  measure  similar  performance  at  SLO. 

The  ESTCP  Study  Plan  (Section  3.0  Performance  Objectives,  Table  3.1)  lists  and  describes 
seven  quantitative  objectives  for  the  classification  study  at  SLO.  We  have  duplicated  that  table 
here  for  convenience  as  Table  3.1.  In  addition  to  those  objectives,  we  included  a  few  additional 
performance  objectives  shown  in  Table  3.2  that  either  supplement  or  expand  the  objectives  in 
Table  3.1. 


Table  3.1:  Global  SLO  Classification  Study  Performance  Objectives 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Data  Collection  Objeci 

tives 

Complete  coverage  of 
the  demonstration  site 

Percentage  of  valid 
points 

Gaps  in  survey 
coverage 

•  Mapped  survey  data 

No  more  than  1%  of 
data  points  have 
spurious  sensor  or 

GPS  readings. 

No  coverage  gap 
larger  than  75  cm 

Repeatability  of 
calibration  strip 
measurements 

Amplitude  of  EM 
anomaly 

Measured  target 
locations 

•  Twice-daily 
calibration  strip 
survey  data 

Amplitudes  ±15% 

Down-track  location 
±25  cm 

Detection  of  all 
munitions  of  interest 

Percent  detected  of 
seeded  items 

•  Location  of  seeded 
items 

•  Anomaly  List 

At  least  98%  of 
seeded  items  detected 

Analysis  and  Classification  Objectives 

Maximize  correct 
classification  of 
munitions 

Number  of  targets-of- 
interest  retained. 

•  Prioritized  anomaly 
lists 

•  Scoring  reports 
from  IDA 

Approach  correctly 
classifies  all  targets- 
of-interest 
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Maximize  correct 
classification  of  non¬ 
munitions 

Number  of  false 
alarms  eliminated. 

•  Prioritized  anomaly 
lists 

•  Scoring  reports 
from  IDA 

Reduction  of  false 
alarms  by  >  30% 
while  retaining  all 
targets  of  interest 

Specification  of  no¬ 
dig  threshold 

Probability  of  correct 
classification  and 
number  of  false 
alarms  at 
demonstrator 
operating  point. 

•  Demonstrator - 
specified  threshold 

•  Scoring  reports 
from  IDA 

Threshold  specified 
by  the  demonstrator  to 
achieve  criteria  above 

Minimize  number  of 
anomalies  that  cannot 
be  analyzed 

Number  of  anomalies 
that  must  be  classified 
as  “Unable  to 

Analyze.” 

•  Demonstrator  target 
parameters 

Reliable  target 
parameters  can  be 
estimated  for  >  90% 
of  anomalies  on  each 
sensor’s  detection  list. 

Correct  estimation  of 
target  parameters 

Accuracy  of  estimated 
target  parameters. 

•  Demonstrator  target 
parameters 

•  Results  of  intrusive 
investigation 

ps  ±  20% 

X,  Y  <15  cm  (la) 

Z  <10  cm  (la) 

size  ±  20% 

Table  3.2:  MetalMapper-Specific  Performance  Objectives  for  the  SLO  Project 


Performance 

Objective 

Metric 

Data  Requirements 

Success  Criteria 

|  Quantitative  Performance  Objectives  || 

Dynamic  Mode 

Survey  Productivity 

Area  surveyed  per  day 

Field  production  log 

>1.2  acre/day  @ 

0.75m 

Lane  spacing 

Repeatability  of 
parameter  estimation 

Recovered  target 
parameters 

Daily  calibrations 
over  canonical  target 
(sphere) 

±10%  on  RMS(Po) 

Static-Mode  Survey 
Productivity 

Static  Pts/day 

Daily  production  logs 

Pts  >30/hr  or 

Pts  >200/day 

3.1.  OBJECTIVE:  COMPLETE  COVERAGE  OF  DEMONSTRATION  SITE 

The  reliability  of  the  survey  data  depends  on  the  extent  of  coverage  of  the  site.  This  objective 
concerns  the  ability  of  the  demonstrator  to  completely  survey  the  site  and  obtain  valid  data. 

3.1.1.  Metric 

The  metrics  for  this  objective  are  the  percentage  of  valid  data  points  and  the  maximum  coverage 
gap  in  the  final,  mapped  data. 
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3.1.2.  Data  Requirements 

A  mapped  data  file  will  be  used  to  judge  the  success  of  this  objective. 

3.1.3.  Success  Criteria 

This  objective  will  be  considered  to  be  met  if  at  least  99%  of  the  mapped  data  points  have  valid 
sensor  and  GPS  readings  and  if  the  maximum  coverage  gap  is  no  more  than  75  cm. 

3.2.  OBJECTIVE:  REPEATABILITY  OF  CALIBRATION  STRIP  MEASUREMENTS 

The  reliability  of  the  survey  data  also  depends  on  the  proper  functioning  of  the  survey 
equipment.  This  objective  concerns  the  twice-daily  confirmation  of  sensor  system  performance. 

3.2.1.  Metric 

The  metrics  for  this  objective  are  the  amplitude  and  down-track  position  of  the  maxima  obtained 
from  each  of  the  twice-daily  surveys  of  the  calibration  strip. 

3.2.2.  Data  Requirements 

The  survey  data  will  be  used  to  judge  this  objective. 

3.2.3.  Success  Criteria 

This  objective  will  be  considered  to  be  met  if  the  measured  amplitudes  for  each  object  are  within 
15%  of  the  mean  and  the  down-track  position  of  the  anomaly  is  within  25  cm  of  the  known 
location. 

3.3.  OBJECTIVE:  DETECTION  OF  ALL  ITEMS  OF  INTEREST 

Quality  data  should  lead  to  a  high  probability  of  detecting  the  munitions  of  interest  at  the  site. 

3.3.1.  Metric 

The  metric  for  this  objective  is  the  percentage  of  seed  items  that  are  detected  using  the  specified 
anomaly  selection  threshold. 

3.3.2.  Data  Requirements 

Each  demonstrator  will  prepare  an  anomaly  list.  IDA  personnel  will  score  the  detection 
probability  of  the  seeded  items. 

3.3.3.  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  at  least  98%  of  the  seeded  items  are  detected. 

3.4.  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  MUNITIONS 

This  is  one  of  the  two  primary  measures  of  the  effectiveness  of  the  classification  approach.  By 
collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms  we  expect  to  be  able  to  classify  the  targets  with  high  efficiency.  This 
objective  concerns  the  component  of  the  classification  problem  that  involves  correct 
classification  of  items-of-interest. 

3.4.1.  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  master  anomaly  list  that  can  be 
correctly  classified  as  munitions  by  each  classification  approach. 
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3.4.2.  Data  Requirements 

Each  demonstrator  will  prepare  a  prioritized  dig  list  for  the  targets  on  the  master  anomaly  list. 
IDA  personnel  will  use  their  scoring  algorithms  to  assess  the  results. 

3.4.3.  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  all  of  the  items-of-interest  are  correctly  labeled  as 
munitions  on  the  prioritized  anomaly  list. 

3.5.  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-MUNITIONS 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter 
estimation  and  classification  algorithms  we  expect  to  be  able  to  classify  the  targets  with  high 
efficiency.  This  objective  concerns  the  component  of  the  classification  problem  that  involves 
false  alarm  reduction. 

3.5.1.  Metric 

The  metric  for  this  objective  is  the  number  of  items-of-interest  on  the  master  dig  list  that  can  be 
correctly  classified  as  non-munitions  by  each  classification  approach. 

3.5.2.  Data  Requirements 

Each  demonstrator  will  prepare  a  prioritized  dig  list  for  the  targets  on  the  master  anomaly  list. 
IDA  personnel  will  use  their  scoring  algorithms  to  assess  the  results. 

3.5.3.  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  more  than  30%  of  the  non -munitions  items  can  be 
correctly  labeled  as  non-munitions  while  retaining  all  of  the  targets-of-interest  on  the  dig  list. 

3.6.  OBJECTIVE:  SPECIFICATION  OF  NO-DIG  THRESHOLD 

In  a  retrospective  analysis  as  will  be  performed  in  this  demonstration,  it  is  possible  to  tell  the  true 
classification  capabilities  of  a  classification  procedure  based  solely  on  the  prioritized  dig  list 
submitted  by  each  demonstrator.  In  a  real-world  scenario,  all  targets  may  not  be  dug  so  the 
success  of  the  approach  will  depend  on  the  ability  of  an  analyst  to  accurately  specify  their 
dig/no-dig  threshold. 

3.6.1.  Metric 

The  probability  of  correct  classification,  Pc]ass,  and  number  of  false  alarms,  Nfa,  at  the 
demonstrator-specified  threshold  are  the  metrics  for  this  objective. 

3.6.2.  Data  Requirements 

Each  demonstrator  will  prepare  a  ranked  anomaly  list  with  a  dig/no-dig  threshold  indicated.  IDA 
personnel  will  use  their  scoring  algorithms  to  assess  the  results. 

3.6.3.  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  more  than  30%  of  the  non-munitions  items  can  be 
correctly  labeled  as  non-munitions  while  retaining  all  of  the  targets-of-interest  at  the 
demonstrator-specified  threshold. 
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3.7.  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  cannot  be  classified  by  the 
classifier.  These  anomalies  must  be  placed  in  the  dig  category  and  reduce  the  effectiveness  of 
the  classification  process. 

3.7.1.  Metric 

The  number  of  anomalies  for  which  reliable  parameters  cannot  be  estimated  is  the  metric  for  this 
objective. 

3.7.2.  Data  Requirements 

Each  demonstrator  that  estimates  target  parameters  will  provide  a  list  of  all  parameters  as  part  of 
their  results  submission  along  with  a  list  of  those  anomalies  for  which  parameters  could  not  be 
reliable  estimated. 

3.7.3.  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  reliable  parameters  can  be  estimated  for  >  90%  of 
the  anomalies  on  each  sensor  anomaly  list. 

3.8.  OBJECTIVE:  CORRECT  ESTIMATION  OF  TARGET  PARAMETERS 

This  objective  measures  the  accuracy  of  the  target  parameters  that  are  estimated  in  the  first  phase 
of  the  analysis.  Successful  classification  is  only  possible  if  the  input  features  are  internally 
consistent.  The  obvious  way  to  satisfy  this  condition  is  to  estimate  the  various  target  parameters 
accurately. 

3.8.1.  Metric 

Accuracy  of  estimation  of  target  parameters  is  the  metric  for  this  objective. 

3.8.2.  Data  Requirements 

Each  demonstrator  that  estimates  target  parameters  will  provide  a  list  of  all  parameters  as  part  of 
their  results  submission.  IDA  analysts  will  compare  these  estimated  parameters  to  those 
measured  during  the  intrusive  investigation  and  determined  via  subsequent  in-air  measurements. 

3.8.3.  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  the  estimated  ps  are  within  ±  20%,  the  estimated 
X,  Y  locations  are  within  15  cm  (la),  the  estimated  depths  are  within  10  cm  (la),  and  the 
estimated  size  is  within  ±  20%. 

3.9.  OBJECTIVE:  DYNAMIC-MODE  SURVEY  PRODUCTIVITY 

Operating  in  a  vehicle-towed  configuration,  we  surveyed  the  entire  test  area  with  the 
MetalMapper  operating  in  its  dynamic  acquisition  mode  using  a  lane  spacing  of  0.75m  and  a 
survey  speed  of  0.5m/s  (30m/min).  Our  objective  was  to  complete  the  dynamic  survey  at  a  rate 
of  1  acre  per  day  or  more. 

The  reviewers  of  this  plan  will  no  doubt  note  that  we  propose  to  perform  our  dynamic  data 
collection  with  0.75m  lane  spacing.  On  the  face  of  it,  therefore,  it  appears  that  we  are  ignoring 
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the  overall  performance  goal  stated  in  5  of  having  no  survey  gaps  greater  than  0.75m.  However, 
the  MetalMapper  antenna  array  contains  7  tri-axial  receiver  cubes  positioned  so  that  each 
receiver  cube  tracks  independent  parallel  profdes  with  an  offset  of  13cm.  Therefore,  the  data 
from  a  single  profile  acquired  with  the  MetalMapper  can  be  split  into  7  profiles  covering  a  cross¬ 
track  swathe  of  0.92m  for  the  purposes  of  generating  a  detection  map.  The  detection  maps  in 
Figure  3.1  were  derived  from  the  same  MetalMapper  data  acquired  over  the  Blind  Grid  at  YPG 
in  June  2008.  The  area  was  surveyed  with  a  lane  spacing  of  lm.  The  map  on  the  left  was 
compiled  by  averaging  a  measure6  of  the  secondary  field  amplitude  of  cube  receivers  2  through 
6  (see  Figure  3.1)  and  thus  simulates  a  map  that  might  be  compiled  from  an  EM-61  (lm  x  lm) 
acquiring  data  with  lm  lane  spacing.  The  map  on  the  right  was  compiled  by  splitting  the  cube 
receivers  into  separate  profiles  and  again  mapping  the  amplitude  of  the  secondary  magnetic  field. 
The  inset  to  the  Split  Cube  map  shows  the  survey  tracks  followed  by  each  of  the  individual 
cubes.  Note  the  there  are  noticeable  gaps  between  groups  of  7  survey  tracks.  These  gaps  are 
representative  of  the  theoretical  16cm  gap  due  to  the  fact  that  our  survey  was  conducted  with  lm 
(nominal)  profiles  while  the  swathe  coverage  of  the  MetalMapper  antenna  array  is  92cm.  When 
viewed  in  this  context  (i.e.,  Split  Cube  map  coverage),  our  survey  with  0.75m  lane  spacing  easily 
meets  the  stated  performance  objective  for  data  gaps. 


6  We  use  a  composite  time  gate  representing  an  average  of  the  gates  spanning  the  interval 
100<t<900ps. 
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“Split  Cube”  Detection 


Figure  3.1:  Detection  maps  derived  from  the  same  MetalMapper  data  set  acquired  over  the 
Blind  Grid  at  YPG.  The  data  were  acquired  with  a  lane  spacing  of  lm.  The  average  5MI 
detection  map  (left)  is  similar  to  the  map  that  would  be  derived  from  an  EM61  survey.  The 
Split  Cube  map  (right)  maps  the  amplitude  of  the  secondary  field  of  each  triaxial  cube 
receiver  independently.  The  result  is  a  detection  map  with  an  effective  Lane  spacing  of  13cm. 
The  inset  shows  the  path  of  the  (split)  cube  receivers 


3.9.1.  Metric 

The  metric  for  dynamic-mode  survey  productivity  will  be  the  average  area  covered  per  day. 

3.9.2.  Data  Requirements 

The  data  files  representing  dynamic  survey  lines  or  static  points  are  tagged  with  both  a  time  and 
a  date.  The  file  name  itself  is  constructed  of  a  unique  text  identifier  to  which  is  appended  a  5- 
digit  index  that  increments  for  each  successive  data  file.  We  index  those  files  daily  on  a 
spreadsheet  which  provides  us  with  a  production  log  that  documents  the  time  it  takes  to  cover 
any  particular  segment  of  the  survey  area  as  well  as  the  total  time  for  completing  the  survey. 
Thus,  we  will  be  able  to  estimate  both  an  average  productivity  for  the  whole  survey  and  on  a 
daily  basis. 

3.9.3.  Success  Criterion 

At  APG,  we  surveyed  approximately  1.2  acres/day  in  the  dynamic  mode.  The  conditions  at  SLO 
will  be  different  and  perhaps  more  challenging  due  to  the  topography  at  SLO.  We  are  planning 
modifications  in  our  antenna  platform,  our  tow  vehicle,  our  navigation  software,  and  our  field 
procedure  with  the  goal  of  exceeding  the  productivity  we  achieved  at  APG. 


ESTCP  Final  Report  MM-0603: 


19 


MetalMapper 


3.10.  OBJECTIVE:  REPEATIBILITY  OF  PARAMETER  ESTIMATION 

This  objective  speaks  to  our  goal  to  demonstrate  that  the  response  of  MetalMapper  system 
remains  stable  over  time.  It  includes  stability  of  the  MetalMapper  hardware  as  well  as  stability 
and  robustness  of  parameter-extraction  processing.  This  objective  is  slightly  different  from  the 
global  objective  “Correct  Estimation  of  Target  Parameters”  (Section  3.7)  in  that  our  static 
calibration  measurements  re-acquire  the  same  target  each  time.  So  this  MetalMapper-specific 
objective  supplements  that  global  objective. 

3.10.1.  Metric 

We  use  a  robust  parameter  extracted  from  the  principal  polarizability  transients  (the  RMS  value 
of  the  three  integrated  polarizability  transients  (Pox,  Poy,  Poz)  estimated  with  our  physics-based 
model.  This  measure  of  system  response  is  independent  of  minor  variations  in  platform  attitude 
and  position  relative  to  a  calibration  target. 

3.10.2.  Data  Requirements 

The  required  data  will  be  acquired  as  part  of  routine  dynamic-mode  and  static-mode 
measurements  over  the  Calibration  Strip  established  at  SLO.  As  indicated  in  the  ESTCP  Study 
Plan  one  of  the  seeded  targets  will  be  a  sphere.7  Our  QC  plan  requires  us  to  visit  the  calibration 
strip  two  or  more  times  per  day.  In  addition,  we  will  have  the  ground-truth  from  the  training  data 
set  (the  first  5  30m  x  30m  blocks  excavated)  and,  eventually,  from  the  rest  of  our  survey.  Using 
the  available  ground-truth  we  will  develop  retrospective  statistics  relating  to  the  reproducibility 
of  the  principal  polarizability  transients  for  each  of  the  target  types  of  interest. 

3.10.3.  Success  Criteria 

We  have  set  our  success  criterion  to  ±10%  based  on  our  experience  from  repeated  static 
calibrations  at  YPG  and  at  APG.  In  the  case  of  the  latter,  we  acquired  a  total  of  35  static 
calibration  measurements  on  our  calibration  target  over  a  period  of  18  days.  The  mean  and 
standard  deviations  for  the  RMS(Po)  extracted  for  those  measurements  was  2369±85  cm3.  Thus, 
the  success  criterion  allows  almost  3  standard  deviations  assuming  that  the  variations  in  the 
parameter  are  Gaussian.  Therefore  we  expect  that  all  of  our  calibration  measurements  will  fall 
within  this  range. 

3.11.  OBJECTIVE:  STATIC-MODE  SURVEY  PRODUCTIVITY 

When  operating  in  the  Cued  ID  or  static  survey  mode  our  objective  is  to  maximize  the  number  of 
sites  visited  per  day  consistent  with  maintaining  data  quality.  Data  quality  is  primarily  a  function 
of  two  variables  in  the  field.  First  it  is  necessary  to  position  the  center  of  the  antenna  platform  as 
nearly  as  possible  to  the  desired  target  point.  Second,  data  quality  depends  on  the  stacking 
time — longer  stacking  time  means  better  data.  Improving  productivity  therefore  means 
decreasing  both  the  time  for  target  reacquisition  and  minimizing  the  stacking  time. 


7  We  assume  that  the  sphere  will  be  located  at  a  depth  that  provides  good  SNR  (>40dB).  If  not, 
we  will  implant  our  own  “Calibration  Sphere.” 
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3.11.1.  Metric 

For  static-mode  surveys,  the  metric  will  be  targets/hr ,8 

3.11.2.  Data  Requirements 

The  data  required  for  measuring  productivity  comes  directly  from  our  field  notes  and  our  index 
of  data  files.  The  data  files  representing  dynamic  survey  lines  or  static  points  are  tagged  with 
both  a  time  and  a  date.  The  file  name  itself  is  constructed  of  a  unique  text  identifier  to  which  is 
appended  a  5-digit  index  that  increments  for  each  successive  data  file.  We  index  those  files  daily 
on  a  spreadsheet.  Measuring  productivity  over  any  period  of  time  is  simply  a  matter  of  counting 
the  number  of  (good)  dynamic  or  static  data  files  over  a  given  period  of  time. 

3.11.3.  Success  Criteria 

Our  criterion  for  success  for  productivity  in  the  static  survey  mode  is  30  targets/hr.  We  achieved 
that  level  of  production  both  at  Yuma  and  at  APG.  With  the  modifications  we  plan  in  our 
procedures  and  navigation  software  as  well  as  in  the  way  we  mount  the  antenna  platform  to 
survey  vehicle,  we  hope  to  improve  on  the  productivity  we  achieved  in  our  other  demonstrations. 


8  We  are  adopting  an  hourly  rate 
significantly  because  of  site  access. 


rather  than  a  daily  rate  here  because  field  days  can  vary 
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4.  SITE  DESCRIPTION 

Over  a  one-year  period,  we  have  conducted  demonstrations  at  the  Standardized  UXO 
Technology  Demonstration  Sites  located  at  Yuma  Proving  Ground  (YPG),  Aberdeen  Proving 
Ground  (APG),  the  Former  Camp  San  Luis  Obispo  in  California  and  most  recently  at  Camp 
Butner  in  North  Carolina.  This  Final  Report  focuses  mostly  on  the  results  of  YPG,  APG,  and 
SLO,  since  the  Camp  Butner  analysis  was  ongoing  at  the  time  this  was  written. 

4.1.  SITE  SELECTION 

4.1.1.  Standardized  Demonstration  Sites  (APG  and  YPG) 

The  UXO  test  sites  at  APG  [11]  and  YPG  [12]  were  specifically  developed  by  the  U.S.  Army 
Environmental  Command  (USAEC)  for  the  testing  of  all  technology  relating  to  the  detection  and 
classification  of  UXO.  Each  of  the  sites  include  an  area  for  calibration  and  training  (Calibration 
Lanes)  where  various  types  of  inert  ordnance  of  interest  have  been  seeded  and  the  ground  truth  is 
made  available  to  demonstrators.  These  sites  include  other  areas  that  have  been  seeded  wherein 
no  ground-truth  is  provided.  Two  types  of  demonstration  areas  are  provided:  1)  grid;  and  2) 
open  field  areas. 

1.  Blind  Grid:  In  the  blind  grid,  targets  have  been  seeded  at  the  centers  of  a  uniform  40x40 
grid  with  grid  centers  at  2m  intervals.  Not  all  grid  cells  are  occupied.  The  blind  grids  are 
designed  to  test  an  instrument’s  sensitivity  for  target  detection  and  discrimination  under 
conditions  where  interference  from  adjacent  targets  (located  a  minimum  of  2m  away)  is 
minimized.  Moreover,  in  a  seeded  cell  the  nominal  (X-Y)  location  of  the  target  is  known. 
We  conducted  demonstrations  and  were  scored  by  the  ATC  over  the  Blind  Grids  at  both 
YPG  and  APG. 

2.  Open  Field  Areas:  In  these  areas,  the  targets  are  seeded  randomly  with  the  same 
standard  targets  seeded  in  the  Calibration  Lanes  and  the  Blind  Grid.  The  MetalMapper 
did  not  demonstrate  over  the  Open  Field  area  at  YPG.  In  2008  and  before  our 
demonstration,  the  Open  Field  Area  at  APG  was  reconfigured  into  two  smaller  areas 
designated  as  the  “Direct  Fire”  (DF)  area  and  the  “Indirect  Fire”  (IF)  area.  Considerable 
effort  was  made  to  seed  the  areas  with  munition  types  (and  clutter)  that  simulate  the 
spatial  distribution  and  target  depths  found  in  direct  and  indirect  fire  impact  zones.  It  is 
significant,  that  the  new  areas  at  APG  were  seeded  with  only  3  types  of  munitions.  The 
DF  was  seeded  with  25mm  M792  (including  brass  casing),  37mm  M63M1  projectile,  and 
105mm  M456  (HEAT)  projectiles.  The  IF  was  seeded  with  60mm  M49A3  and  81mm 
M374  mortar  rounds,  and  the  105mm  M60  artillery  projectile. 

4. 1 .2.  Camp  San  Luis  Obispo  (SLO) 

The  former  camp  San  Luis  Obispo,  located  near  San  Luis  Obispo,  CA  was  carefully  selected  by 
the  ESTCP  program  office  as  the  demonstration  site  for  classification  study  to  be  conducted  in 
2009.  The  MetalMapper  was  one  of  3  advanced  electromagnetic  systems  to  participate  in  that 
study.  The  12-acre  study  area  is  a  former  mortar  and  2.36-in  bazooka  rocket  range.  There  site 
contains  4  munitions  types  of  primary  interest:  2.36-in  rockets,  60mm  mortars,  81mm  mortars, 
and  4.2-in  mortars.  In  contrast  with  the  standardized  UXO  test  areas  at  YPG  and  APG,  the  SLO 
demonstration  area  is  situated  on  a  hill  and  as  such  presents  a  significant  challenge  for  survey  in 
both  detection  or  survey  mode  acquisition  and  in  cued  ID  or  static  mode  acquisition. 
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4.2.  SITE  HISTORY 

The  standardized  sites  program  is  a  multi-agency  undertaking  being  coordinated  by  the  U.S. 
Army  Environmental  Command  with  support  provided  by  the  Aberdeen  Test  Center  (ATC),  and 
the  Army  Corps  of  Engineers.  Funding  is  provided  by  the  Army  Environmental  Quality 
Technology  Program  (EQTC),  ESTCP  and  SERDP.  The  standardized  sites  at  APG  and  YPG 
were  constructed  at  two  major  army  testing  facilities  at  either  end  of  the  country.  Each  of  these 
installations  has  a  long  history  of  use  for  the  testing  and  evaluation  of  military  munitions.  Each 
site  meets  other  requirements  for  siting  of  a  UXO  Demonstration  area  that  have  been  enumerated 
in  a  “handbook”  that  outlines  the  procedure  for  selecting  and  constructing  such  demonstration 
sites  [13].  The  sites  at  APG  and  YPG  were  completed  in  2002.  Since  then  each  site  has  hosted 
dozens  of  partial  or  full  demonstrations  [14], 9  The  APG  site  has  been  reconfigured  3  times,  most 
recently  in  2008.  The  YPG  site  is  currently  on  its  second  reconfiguration. 

Camp  San  Luis  Obispo  was  established  in  1928  as  a  facility  for  the  California  National  Guard.  It 
was  used  extensively  during  World  War  II  as  a  training  facility  and  again  during  the  Korean 
conflict.  The  facility  was  inactive  after  the  Korean  conflict.  At  the  present  time,  approximately 
half  of  the  original  land  has  been  transferred  to  Cal  Poly  State  University  and  Cuesta  College  and 
the  other  half  has  been  retained  by  the  California  National  Guard. 

4.3.  SITE  GEOLOGY 

4.3.1.  Standardized  Sites 

The  soil  at  the  APG  site  consists  of  low  permeability  poorly  drained  soil  with  relatively  high 
moisture  content  (20%-30%).  Topography  is  “flat”  with  maximum  slopes  of  l%-2%.  The  soil 
at  YPG  is  typical  alluvium  characteristic  of  basin  and  range  pediment  to  be  found  throughout  the 
desert  southwest.  The  soil  has  low  moisture  content  (typically  <  7%).  Moisture  increases  with 
depth  and  moisture  levels  greater  than  3%  are  only  found  at  depths  greater  than  lm.  These  soils 
have  relatively  low  magnetite  content  with  volume  magnetic  susceptibilities  typically  around  100 
SI.  As  with  the  APG,  the  general  topography  is  flat. 

4.3.2.  San  Luis  Obispo 

In  contrast  to  the  standardized  test  sites,  topographic  relief  over  the  12-acre  SLO  site  is  in  excess 
of  200  ft  with  slopes  sometimes  exceeding  15%.  Soil  textures  and  thicknesses  range  widely. 
Thick  sandy  alluvium  is  found  adjacent  to  washes  and  drainage  channels.  Thin  to  thick 
mountain  terrace  soils  typically  overlay  crystalline  metamorphic  bedrock  and  are  usually 
comprised  of  sandy  to  silty  loam  with  a  thin  veneer  of  silty  clay  near  the  surface.  Moisture 
levels  vary  seasonally.  At  the  time  of  our  demonstration  (May- June  2009),  the  soils  were  very 
dry. 


9  A  partial  demonstration  is  here  defined  as  conducting  a  minimal  survey  consisting  of  the 
Calibration  Lanes  plus  the  Blind  Grid.  A  full  demonstration  requires  a  survey  over  the  Open 
Field  area. 
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4.4.  MUNITIONS  CONTAMINATION 

4.4.1.  Standardized  Sites 

The  YPG  site  has  been  seeded  with  a  diverse  set  of  inert  munitions  ranging  from  20mm  M55 
projectiles  on  the  small  end  to  155mm  M483Alprojectiles  on  the  large  end.  The  munitions  types 
include  a  number  of  common  submunitions.  The  calibration  lanes  have  also  been  seeded  with  a 
few  (uncommon)  non-munition  targets  such  as  shot  puts,  steel  disks,  and  shorted  conducting 
loops.  A  complete  description  of  munitions  types  to  be  found  at  YPG  and  APG  can  be  found  by 
following  appropriate  links  located  on  Standardized  UXO  Technology  Demonstration  Program 
web  site  [10]. 

The  APG  demonstration  site  was  reconfigured  in  early  2008.  As  part  of  that  reconfiguration,  the 
open  field  was  sub-divided  into  a  “Direct  Fire”  (DF)  area,  and  an  “Indirect  Fire”  (IF)  area.  Each 
of  these  areas  has  been  seeded  with  3  munitions  types  that  are  appropriate  to  the  type  of  range 
they  simulate.  The  DF  contains  3  types  of  direct  fire  munitions:  a)  25mm  rounds;  b)  37mm 
projectiles;  and  c)  105mm  HEAT  projectiles.  Similarly,  the  IF  contains  3  types  of  indirect  fire 
munitions:  a)  60mm  mortars;  b)  81mm  mortars;  and  c)  105mm  artillery  projectiles.  The  newly 
reconfigured  Blind  Grid  at  APG  contains  6  munitions  types,  the  3  types  seeded  in  the  DF  area 
and  the  3  types  seeded  in  the  IF  area.  The  Calibration  Lanes  at  APG  contain  more  than  the  6 
munition  types  seeded  in  the  Blind  Grid,  DF,  and  IF.  However,  the  target  density  has  been 
significantly  decreased  in  an  effort  to  reduce  interference  between  targets.  So  there  are  fewer 
targets  overall  and  the  main  emphasis  has  been  to  provide  demonstrators  with  opportunities  to 
characterize  the  6  munition  types  that  have  been  buried  in  the  Blind  Grid,  the  IF,  and  the  DF. 

4.4.2.  San  Luis  Obispo 

The  San  Luis  Obispo  site  is  a  live  site.  A  preliminary  dig  of  two  small  50  ft  x  50  ft  grids  in  the 
12-acre  area  of  interest  revealed  4  types  of  munitions:  2.36-in  rockets,  60mm  mortars,  81mm 
mortars,  and  4.2-in  mortars. 
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5.  TEST  DESIGN 

5.1.  CONCEPTUAL  EXPERIMENTAL  DESIGN 

Each  of  the  field  demonstrations  was 
conducted  in  3  phases: 

1 .  Dynamic-mode  survey  for  target 
detection 

2.  Static-mode  survey  for  “Cued-ID” 
discrimination 

3.  Post-acquisition  analysis 

The  Gantt  charts  in  Figure  5.1  and  5.2 
provide  time-lines  for  each  of  the 
demonstrations  and  also  serve  as  an  outline 
for  the  various  activities  involved. 

5.1.1.  Target  Detection  (Mapping 
Survey) 

In  the  first  phase  of  each  field 
demonstration,  the  MetalMapper  is  operated 
in  its  dynamic  (mapping)  mode  in  order  to 
generate  a  detection  map.  In  its  dynamic 
mode,  only  the  horizontal  (Z)  transmitter 
loop  is  energized.  Data  are  acquired  along 
a  series  of  parallel  profiles.  These  data  are 
compiled  into  a  detection  map  wherein  the 
mapped  parameter  indicates  a  peak  centered 
directly  over  any  metallic  target  within 
range.  Peaks  are  chosen  (i.e.  “detected”) 
either  manually  using  the  computer  mouse 
or  automatically  using  an  automatic  peak 
detector  that  operates  on  a  data  grid.  In  all 
cases  where  an  automatic  peak  detector  is 
employed,  it  is  necessary  to  edit  the 
resulting  peaks  in  order  to  remove  or 
“merge”  multiple  target  picks  on  the  same 
anomaly.  The  resulting  edited  target  list  is 
used  during  the  next  (Cued  ID)  phase  of  the 
demonstration. 

5.1.2.  Cued  ID  (Static  Survey) 

The  target  list  generated  from  the  mapping 
survey  “cued”  (sequenced)  so  that  each 
target  can  be  efficiently  reacquired. 

Reacquisition  requires  that  the  antenna  platform  be  nominally  centered  over  the  anomaly  of 
interest.  Once  a  target  is  reacquired,  the  MetalMapper  is  activated  to  acquire  a  single  static  data 
set.  For  static  data  acquisition,  each  of  the  three  transmitter  loops  is  energized  while  data 


A)  Activities  -  MetalMapper  YPG  Demonstration  (June-July  2008) 


ID 

Task  Name 

Start 

Finish 

Duration 

Jun  2008  |  Jul  2008 

m  6115  |  6/22  |  6/29  |  7/6 

1 

Mobilization 

6/7/2008 

6/8/2008 

2d 

a 

2 

Setup  &  Det.  Survey  -  Calibration  Grid 

6/9/2008 

6/9/2008 

Id 

± 

3 

Cued  ID  -  Calibration  Grid 

6/10/2008 

6/10/2008 

75d 

4 

Cued  ID  -  Blind  Grid 

6/10/2008 

6/11/2008 

Id 

pb 

5 

Det  Survey  -  Blind  Grid 

6/11/2008 

6/12/2008 

1.25d 

6 

Det  Experiments 

6/13/2008 

6/13/2008 

.5d 

7 

Field  Processing  &  QA/QC 

6/9/2008 

6/13/2008 

5d 

□ 

8 

Demobilization 

6/13/2008 

6/14/2008 

1.5d 

9 

Post-Acquisition  Processing 

6/15/2008 

7/11/2008 

27d 

B)  Activities  -  MetalMapper  APG  Demonstration  (Sep-Oct  2008) 


C)  Activities  -  MetalMapper  SLO  Demonstration  (May-Jun  2009) 


ID 

Task  Name 

Start 

Finish 

Duration 

May  2009  |  Jun  2009 

|  5/24  |  5/31  |  6/7  |  6/14  |  6/21  | 

1 

Mobilization 

5/23/2009 

5/24/2009 

2d 

□ 

2 

Setup 

5/25/2009 

5/25/2009 

.5d 

± 

3 

Det  Survey 

5/25/2009 

6/1/2009 

7.5d 

4 

Static  -  Tst  Pit  &  Tst  Strip 

6/3/2009 

6/3/2009 

.5d 

5 

Break 

6/4/2009 

6/7/2009 

4d 

tb 

6 

Static  Tests 

6/8/2009 

6/8/2009 

.5d 

7 

Cued  ID  Survey 

6/8/2009 

6/15/2009 

7d 

8 

Repeat  Dyn  Survey 

6/15/2009 

6/16/2009 

.5d 

& 

9 

Repeat  Static  Survey 

6/16/2009 

6/16/2009 

Id 

10 

Take  Down/Demob 

6/17/2009 

6/18/2009 

2d 

•hi 

11 

Field  Processing  QA/QC 

5/25/2009 

6/17/2009 

24d 

I  1 

Figure  5.1:  Gantt  charts  providing  time  lines 
and  activity  breakdowns  for  3  MetalMapper 
demonstrations  covered  in  this  report. 
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transients  are  measured  from  the  7  triaxial  cube  receivers  (21  transients).  Because  the  antenna 
platform  is  motionless,  each  receiver  transient  can  be  repetitively  stacked  over  many  cycles  of 
the  transmitter  thus  significantly  reducing  incoherent  random  noise.  A  complete  data  set  that 
includes  the  results  from  energizing  each  of  the  three  transmitter  loops  takes  25-50  seconds 
depending  on  the  stacking  parameter. 

5.1.3.  Post- Acquisition  Analysis 

Preliminary  data  processing  for  the  purpose  of  generating  detection  maps  and  for  QA/QC 
purposes  is  carried  out  throughout  the  course  of  the  field  demonstration.  However,  most  of  the 
data  processing  required  to  assemble  a  target  list  for  scoring  occurs  after  the  field  activities  have 
been  completed  (post-acquisition).  Post-acquisition  processing  includes  the  following  steps: 

1.  Target  Association:  Static  data  points  are  associated  with  a  particular  target  site  on  the 
basis  of  proximity.  When  there  is  more  than  one  static  data  point  associated  with  a 
particular  target  site,  a  decision  is  made  as  to  whether  the  files  are  associated  with  a 
single  target  or  multiple  targets. 

2.  Feature  Selection:  Statistical  analyses  including  scatter  plots,  principal  component 
analysis,  and  trial  and  error  are  performed  to  help  select  a  set  of  features  {feature  vector ) 
associated  with  the  targets  that  we  think  best  identifies  the  targets  of  interest  from 
clutter. 

3.  Library  Matching:  Generate  a  library  of  “type”  polarization  curves  representing  the 
munitions  types  known  to  exist  in  the  area  of  interest.  Match  the  polarization  curves 
extracted  from  each  of  the  unknown  targets  with  those  in  the  type  library.  Select  the 
target  type  with  the  best  matching  score. 

4.  Neural  Network  Training:  Using  feature  vectors  corresponding  to  targets  for  which 
ground  truth  is  available  (e.g.,  from  free-air  static  measurements,  test  lanes  and 
instrument  verification  strips,  and  training  digs),  train  a  neural  network. 

5.  Neural  Network:  Apply  feature  vectors  from  each  target  in  the  test  set  (i.e.,  all 
unclassified  targets)  to  the  neural  network  trained  in  step  4.  The  neural  network  output 
represents  the  primary  classifier. 

6.  Target  List  Assembly:  In  the  final  processing  step,  the  results  from  both  library 
matching  and  neural  network  analysis  are  merged  to  form  a  single  classification  score. 
Visual  QC  is  performed  on  the  prioritized  list  using  cataloged  polarization  curves. 

5.2.  SITE  PREPARATION 

The  standardized  UXO  sites  have  been  “sanitized”  and  then  seeded  with  inert  munitions 
according  to  procedures  reported  in  a  handbook  assembled  as  part  of  the  Standardized  UXO 
Demonstration  Program  [13].  Unlike  the  standardized  test  sites  at  APG  and  YPG,  Camp  San 
Luis  Obispo  is  a  live  site.  Preliminary  investigations  included  a  magnetometer  transect  survey 
and  an  EMI  survey  over  a  larger  area  to  assist  in  selecting  a  smaller  site  suitable  for  the 
discrimination  study.  In  addition,  2  small  50ft  x  50ft  grids  were  dug  in  order  to  identify  the 
munition  types  that  would  likely  be  encountered  during  the  SLO  study.  Finally,  additional 
munitions  of  the  type  expected  at  SLO  were  seeded  in  order  to  improve  the  ratio  of  targets-of- 
interest  to  clutter  items. 
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5.3.  SYSTEM  SPECIFICATION 

The  prototype  MetalMapper  conforms  to  our  present  vision  of  the  Cued  ID  system.  Since  the 
Mapping  system  is  a  subset  of  that,  we  are  also  able  to  use  the  Cued- ID  system  for  mapping. 

5.3.1.  Antenna  Platform 

The  MetalMapper’s  three  transmitting  loops  are  positioned  as  follows: 

•  Z  transmitter  (coil  axis  vertical):  lm  x  lm,  center  is  ~15cm  above  ground  level.  The 
center  of  the  Z  loop  is  taken  to  be  the  local  origin  of  coordinates  for  the  cart. 

•  Y  transmitter  (coil  axis  horizontal  in  direction  of  travel):  lm  x  lm,  centered  0.56m  above 
the  origin. 

•  X  transmitter  (coil  axis  horizontal,  clockwise  from  Y):  0.98m  x  0.98m,  centered  0.56m 
above  the  origin. 

The  MetalMapper’s  seven  receivers  are  positioned  as  shown  in  Figure  2.6.  Note  that  the  seven 
receivers  traverse  profiles  that  are  13cm  apart  in  the  cross-track  (x)  dimension. 

5.3.2.  Signals  and  Timing 

The  MM  system  is,  by  design,  a  very  flexible  system  for  acquisition  of  time  domain  EM  (TEM) 
data.  It  is  beyond  the  scope  of  this  document  to  fully  describe  that  flexibility.  Simply  stated, 
data  are  acquired  in  time  blocks  that  consist  of  a  fixed  number  of  transmitter  cycle  “Repeats”. 
Both  the  period  (T)  and  the  repeat  factor  (N)  are  operator  selectable  and  are  varied  in 
multiplicative  factors  of  3. 10  The  MM  also  averages  an  operator-specified  number  of  acquisition 
blocks  (NStack)  together  before  the  acquired  data  are  saved  to  disk. 

The  transmitter  is  energized  with  a  bipolar  waveform  as  illustrated  in  Figure  2.7.  Therefore  the 
alternating  decay  transients  received  during  the  off  times  must  be  synchronously  rectified  and 
stacked  (averaged).  Decay  transients  in  each  acquisition  block  are  first  rectified  and  stacked  and 
then  averaged  with  other  the  resulting  stacked  transients  from  successive  acquisition  blocks 
(assuming  the  operator  has  selected  NStack  greater  than  one).  The  resulting  data  are  saved  as  a 
data  point.  All  GPS  readings  received  during  the  acquisition  period  are  averaged  together.  If  no 
GPS  readings  are  acquired  during  that  period,  the  most  current  GPS  position  and  the  platform 
attitude  angles  (magnetic  heading,  pitch,  and  roll)  are  used.  GPS  points  are  non-synchronously 
acquired  at  20  points  per  second.  Therefore,  depending  on  relative  timing,  GPS  coordinates  that 
are  stored  with  the  data  are  either  the  most  recent  fix,  or  the  average  of  fixes  received  during  the 
data  point  collection  interval.  By  varying  the  block  period  (T)  and  the  repeat  factor  (R)  settings, 
we  can  set  the  sample  rate  as  high  as  30  data-points/sec  or  as  low  as  desired  (  e.g.  ~1  data-point 
per  minute  typically  for  static  data  points. 

Timing  of  all  EM  signals  including  transmitter  switching  and  receiver  sampling  is  controlled  by 
hardware  that  is  under  the  control  of  the  acquisition  software.  The  MetalMapper  collects  data  in 


10  The  options  for  N  (3,9,27,...)  and  T  =  (0.0333,  0.1,  0.3,  0.9s)  are  such  that  the  resulting 
fundamental  transmitter  frequency  and  its  odd  harmonics  do  not  coincide  with  60Hz  and  its  odd 
harmonics. 
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a  so-called  double-buffered  mode  meaning  that  the  data  samples  from  a  previously  collected  data 
point  are  processed  and  stored  concurrently  with  acquisition  of  the  next  data  point.  If  computer 
processing  cannot  maintain  pace,  one  or  more  succeeding  data  points  are  skipped.  For  normal 
surveying  activities,  data  points  are  rarely  skipped. 

The  MM  has  two  data  acquisition  modes:  Single-Point-Mode  and  Continuous-Mode.  Data 
collection  and  processing  is  the  same  for  either  mode.  In  single-point  mode  the  system  collects  a 
data  point  as  previously  described  and  then  terminates  acquisition.  The  data  are  stored  as  a 
single  data  point  in  the  output  data  file.  In  continuous  mode,  the  system  initiates  collection  of  a 
new  data  point  concurrently  with  completion  of  the  previous  data  point  and  continues  until  the 
operator  intervenes.  All  of  the  data  points  are  stored  to  the  same  output  data  file. 

Once  acquired,  the  decay  transients  in  a  data  point  are  decimated  into  a  set  of  logarithmically 
spaced  time  gates.  Received  signals  are  sampled  at  a  rate  of  250  KS/s.  After  initiating  turn-off 
of  the  transmitter,  the  system  initiates  a  time  delay  (e.g.,  100ps)  determined  by  the  HoldOff 
parameter.  Thereafter,  digital  samples  falling  within  a  specific  time  gate  are  averaged  and 
become  the  value  of  the  signal  for  that  time  gate.  The  widths  of  the  gates  are  specified  by  the 
Gate  Width  parameter.  Gate  width  is  specified  as  a  percentage.  A  gate  width  parameter  of  10%, 
for  example,  will  generate  a  gate  of  width  60  ps  for  a  delay  time  of  600  ps.  The  minimum  gate 
width  is  one  data  sample  (4  ps). 

5.3.3.  Data  Acquisition:  Dynamic  Data 

To  collect  dynamic  data,  the  MM  is  operated  in  Continuous  mode  and  all  data  from  a  single 
survey  line  are  stored  in  a  single  file 

5.3.4.  Post- Acquisition  Data  Processing:  Dynamic  Data 

Data  are  stored  as  binary  formatted  files.  The 
processing  software  includes  the  capability  to 
export  the  data  to  a  Geosoft  Oasis  Montaj  data  base 
and/or  to  text  files.  For  dynamic  surveys,  Oasis 
Montaj  is  used  for  display,  QC,  map  compilation, 
etc. 

As  we  indicated  in  Section  3  of  this  report,  we 
follow  a  two-step  approach  to  target 
characterization.  The  first  step  is  target  detection 
and  for  that  we  conduct  a  dynamic  survey.  Using  a 
custom  Geosoft  executable  (GX),  the  data  from  the 
dynamic  survey  are  imported  into  a  Geosoft  data 
base  (GDB).  Other  custom  GX’s  have  been 
developed  to  process  the  transient  data  to  a  point 
where  they  can  be  further  processed  and  displayed 
using  standard  capabilities  within  the  Geosoft 
Montaj  software  package.  The  end  products  of  the 
post-acquisition  processing  of  the  dynamic  data  are 
target  detection  maps  and  profiles  and  a  list  of 
detected  targets  along  with  their  respective  X-Y 
coordinates.  In  Figure  5.2,  we  show  the  detection 


Figure  5.2:  Target  detection  map  - 
Blind  Grid.  MetalMapper 
demonstration  at  APG  (Sep  ’08).  The 
“+”  symbol  denotes  the  center  of  a  2m 
x  2m  cell. 
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map  that  we  generated  from  dynamic  data 
acquired  over  the  Blind  Grid  at  APG. 

5.3.5.  Post-Acquisition  Processing:  Static 
Data 

Static  data  points  are  acquired  with  the 
MetalMapper  platform  located  over  each  of 
the  targets  identified  using  the  dynamic  data 
as  described  above.  These  data  files  are 
processed  with  our  physics-based  modeling 
program  MM/RMP  to  extract  target-specific 
parameters.  An  example  of  the  resulting 
polarizability  plot  is  shown  in  Figure  5.3. 

Figure  5.3:  Principal  polarizability  transients 
5.4.  CALIBRATION  ACTIVITIES  output  from  MM/RMP. 

Calibration  checks  of  the  MetalMapper 

system  were  performed  at  least  two  times  per  day  as  part  of  our  routine  QA/QC  procedure.  At 
YPG  and  APG,  these  checks  were  conducted  over  a  calibration  sphere  (“CalBall”)  buried  in  an 
area  having  low  background  response.  At  SLO,  the  dynamic  calibration  was  performed  over  the 
Calibration  Strip.  A  check  on  the  static  measurement  was  performed  using  a  steel  sphere  located 
in  the  calibration  pit.  The  calibration  procedure  is  enumerated  below: 

1 .  Acquire  a  static  background  data  set  over  a  designated  “background”  point  (typically 
a  designated  point  at  the  beginning/end  of  the  Calibration  Strip.  Acquisition 
parameters  are  set  at  values  for  static  ( Cued  ID)  data  acquisition  (section  5.5,  Table 

7). 

2.  Acquire  dynamic  data  over  calibration  profile  in  reciprocal  directions  using  our 
standard  dynamic  survey  acquisition  parameters  (see  Section  5.5). 

3.  Acquire  a  static  data  set  with  the  antenna  platform  centered  over  a  designated 
Calibration  Target.11 

The  calibration  procedure  provides  all  the  data  required  to  perform  a  variety  of  QA/QC  checks 
that  document  that  the  instrument  is  functioning  correctly.  Moreover,  the  resulting  data  will  be 
used  to  establish  the  long-term  stability  of  the  instrument  response. 


11  We  use  a  steel  sphere  buried  in  an  area  of  low  background.  At  YPG  and  APG,  the  Calibration 
Sphere  was  used  for  both  static  and  dynamic  calibration.  At  SLO,  we  used  the  Test  Strip  for 
dynamic  calibration.  A  steel  sphere  located  in  the  Test  Pit  at  SLO  was  used  for  static 
calibrations. 
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5.5.  DATA  COLLECTION 


The  MetalMapper  DAQ  has  two  modes  of  data  collection,  dynamic  and  static.  Data  collected  in 
dynamic  mode  results  in  data  files  containing  many  data  samples.  Generally  speaking,  dynamic 
mode  data  are  collected  while  the  antenna  platform  is  in  motion.12  Static  mode  data  collection  is 
employed  for  Cued  ID  surveys.  As  its  name  implies,  the  antenna  platform  remains  static  or 
motionless  during  the  period  of  data  acquisition.  Depending  on  the  acquisition  parameters  (i.e., 
sample  period,  and  stacking  parameter)  it  can  take  10’s  of  seconds  to  complete  a  static 
measurement.  The  results  of  the  static  measurement  are  written  into  a  binary  data  file  containing 
only  a  single  data  point  representing  the  average  (stacked)  result  usually  over  10’s  or  even  100’s 
of  repetitions  of  the  transmitter’s  base  frequency.  In  Table  5.1  below,  we  tabulate  the  acquisition 
parameters  that  we  used  during  the  bulk  of  the  demonstration.  The  5  shaded  columns  relate  to 
parameters  that  can  be  set  by  the  operator.  Those  5  parameters  relate  to  more  universally 
understood  acquisition  parameters  such  as  base  frequency,  transient  decay  length,  and  sample 
period/rate.  We  will  describe  the  following  in  terms  of  the  more  familiar  unshaded  columns. 
These  parameters  were  chosen  with  the  benefit  of  some  experimentation  over  the  course  of  our 
demonstrations  at  YPG  and  APG  in  2008.  At  SLO,  we  decided  to  acquire  our  static  data  with 
8,333ps  decay  transients  based  on  the  premise  that  the  improved  data  quality  arising  from 
stacking  3  times  more  transients  was  preferable  to  acquiring  poorer  quality  transients  over  an 
extended  time  bandwidth  (8,333  ps  versus  25,000  ps).  Note  that  the  Sample  Period  shown  in  the 
table  is  for  the  stack  constant  shown  -  for  static  data,  the  sample  period  can  be  adjusted  as 
desired  by  choosing  a  different  stack  constant.  At  YPG  and  APG  we  used  a  stack  constant  of  20 
resulting  in  a  sample  period  of  54  seconds.  At  SLO,  we  chose  a  stack  constant  of  10  because 
most  targets  were  shallow  and,  in  general,  the  signal-to-noise  ratio  was  always  high. 


Table  5.1:  Acquisition  parameters  used  during  MetalMapper  demonstrations 


Mode 

Hold-Off 
Time  (us) 

Block 

Period 

(s) 

Rep 

Fctr 

Dec 

Fctr 

(%) 

Stk 

Const 

Base 

Freq 

(Hz) 

Decay 
Time  (us) 

No. 

Gates 

Sample 
Period  (s) 

Sample 
Rate  (S/s) 

Dynamic 

50 

0.1 

27 

5 

1 

270 

926 

58 

0.1 

10 

Static 

50 

0.9 

27 

10 

10/20 

30 

8333 

50 

27/54 

N/A 

YPG  Demonstration 

The  objective  of  the  YPG  demonstration  was  to  assess  our  readiness  to  conduct  a  full 
demonstration  at  APG  later  that  year.  Consequently,  we  only  allocated  5  field  days  to  the  effort 
and  confined  our  survey  activities  to  the  Calibration  Grid  and  Blind  Grid  areas  (see  Figure  5.1  A). 
These  activities  included  both  dynamic  and  cued  ID  surveys  and  allowed  us  to  submit  the 
Detection  Stage  and  Discrimination  Stage  target  lists  required  by  the  ATC  as  a  condition  of 
access  to  the  site. 


12  We  also  use  dynamic-mode  data  collection  when  we  want  to  study  noise  statistics  for  statically 
acquired  data. 
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A  PCI  Demonstration 

The  demonstration  at  APG  was  our  first  full  demonstration.  Both  standardized  demonstration 
sites  (APG  and  YPG)  are  operated  under  the  same  rules.  And  therefore,  we  were  required  to 
survey  the  Calibration  Lanes  and  the  Blind  Grid.  In  any  case,  these  surveys  were  necessary  and 
desirable  because  the  APG  site  had  been  reconfigured  earlier  in  2008  and  the  reconfiguration 
reduced  the  targets  of  interest  to  6.  In  addition  to  the  Blind  Grid,  we  also  surveyed  the  newly 
configured  “Indirect  Fire”  and  “Direct  Fire”  areas.  These  two  areas  each  cover  approximately  1 
ha  and  are  located  within  the  confines  of  the  former  “Open  Field”  area.  For  budgetary  reasons, 
we  did  not  survey  the  remainder  of  the  Open  Field  (now  referred  to  as  the  “Legacy”  area). 

SLO  Demonstration 

We  surveyed  the  entire  1 1.8  acre  study  area  at  SLO  with  the  MetalMapper  operating  in  detection 
mode.  Using  maps  from  these  data,  we  selected  2178  targets  for  cued  ID  measurements.  We 
also  repeated  cued  ID  measurements  over  314  of  these  targets  based  on  inversion  results  that 
indicated  that  the  horizontal  offset  of  these  targets  was  greater  than  0.4m.  During  post¬ 
acquisition  processing,  we  were  able  to  reduce  the  number  of  target  picks  that  we  submitted  to 
the  program  office  for  the  following  reasons: 

1.  Target  Detection  Threshold  Adjustment:  The  SLO  dynamic  survey  was  conducted 
using  two  different  platform  deployment  methods:  wheel-mounted,  and  skid-mounted. 
The  antenna  height  above  ground  level  for  wheel-mounted  deployment  was  8cm  higher 
than  for  skid-mounted  deployment.  The  detection  threshold  was  originally  set  for  the 
wheel-mounted  deployment  and  was  not  changed  when  we  reverted  to  skid-mounted 
deployment.  When  we  adjusted  the  target  detection  threshold  upward  for  areas  surveyed, 
we  were  able  to  remove  approximately  300  targets  from  our  target  list. 

2.  Removal  of  Targets  Falling  within  Defined  Areas:  At  the  direction  of  the  program 
office,  all  targets  falling  within  the  two  defined  areas  were  removed:  a)  Vehicle  Only 
Area;  and  b)  Vehicle  Access  Road.  These  areas  and  the  targets  falling  within  them  were 
removed  in  order  to  reduce  the  total  number  of  targets  to  be  dug  for  validation. 

3.  Target  Merges:  We  had  several  requests  from  the  IDA  to  examine  target  groups  in 
order  to  determine  whether  they  related  to  a  single  buried  item  or  multiple  buried  items. 
Moreover,  our  own  post-acquisition  analysis  identified  other  target  groups.  This 
processing  allowed  us  to  merge  a  number  of  closely  spaced  target  picks  into  a  single  pick 
with  2  or  more  associated  MM  static  data  files.  There  were  314  repeated  measurements 
that  fall  in  this  class.  But  there  were  also  double  picks  (i.e.,  2  peaks  on  the  same 
anomaly)  and  picks  on  the  same  anomaly  at  the  boundary  of  adjacent  sub-areas. 

4.  Targets  Picks  with  No  Associated  MM  Static  File:  There  were  21  targets  on  the 
detection  list  that  we  submitted  to  the  program  office  with  no  associated  MM  static  file. 
With  one  exception,  these  targets  were  targets  near  their  associated  detection  thresholds. 
They  are  present  as  a  result  of  the  need  to  re-grid  the  dynamic  survey  and  re-pick  the 
targets  after  IDA  noted  a  systematic  offset  bias  between  the  location  of  our  target  picks 
and  the  location  of  seeded  targets.  After  some  investigation,  we  found  that  the  bias  was 
the  result  of  a  bug  in  the  GX  that  computed  the  slope  correction. 

We  ended  up  submitting  a  detection  target  list  containing  a  total  of  1638  target  picks.  We  were 
able  to  associate  at  least  one  MM  static  file  with  1617  picks. 
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5.5  VALIDATION 


The  YPG  and  APG  test  sites  are  seeded  sites  that  have  been  constructed  and  reconfigured  from 
time  to  time  using  protocols  that  have  been  documented  by  ATC  [13].  Each  demonstrator  is 
required  to  submit  target  lists  for  both  detection  and  discrimination.  In  conformance  with  the 
ATC  protocol,  we  have  submitted  target  lists  for  those  areas  surveyed  with  the  MetalMapper  at 
both  YPG  and  APG.  ATC  has  issued  formal  scoring  reports  for  both  of  those  submissions  [15, 
16]. 

Camp  SLO  is  a  live  site.  Detected  items  have  been  dug  by  Parsons,  Inc  under  contract  with  the 
ESTCP.  Parsons  utilized  standard  procedures  for  digging  UXO  live  sites.  In  addition,  the 
position,  depth,  approximate  attitude,  and  a  photograph  of  the  resulting  target  found  were 
recorded.  The  results  of  the  first  5  30m  x  30m  survey  blocks  were  provided  to  all  data 
processing  demonstrators  for  use  as  training  data.  The  remainder  of  the  dig  results  were  held  in 
confidence  by  the  program  officer  and  IDA  and  used  for  the  purpose  of  scoring  discrimination 
target  lists  prepared  by  the  demonstrators  according  to  protocols  described  in  a  scoring 
memorandum  prepared  by  the  IDA  and  distributed  to  the  demonstrators  [16]. 
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6.  DATA  ANALYSIS  AND  PRODUCTS 

6.1.  PREPROCESSING 

6.1.1.  Dynamic  Survey  Data 

The  dynamic  survey  data  consists  of  parallel  profiles  acquired  with  0.75m  offsets.  Each  profile 
is  recorded  to  a  separate  binary  data  file.  From  these  data,  we  will  compile  detection  parameter 
maps  of  the  surveyed  area  or  sub-area.  The  detection  map  is  based  on  the  magnitude  of  the 
secondary  fields  measured  at  each  of  the  7  tri-axial  receiver  sensors.  The  following  processing 
steps,  accomplished  using  Geosoft  Oasis  Montaj  (OM),  are  required: 

1.  MetalMapper  data  are  recorded  as  binary  files.  These  data  are  imported  directly  into  an 
OM  database  where  simple  editing  is  performed  (e.g.,  editing  line  numbers,  deselecting 
duplicate  lines,  trimming  and  deleting  bad  data  or  stops,  etc).  All  other  steps  are 
accomplished  from  within  OM  using  its  standard  editing  and  processing  capabilities 
supplemented  where  necessary  with  custom  Geosoft  Executables  (GX’s)  and  Geosoft 
Scripts  (GS’s)  and  Geosoft  mathematical  expression  (EXP)  files. 

2.  Convert  Lat/Lon  to  UTM  coordinates 

3.  Make  a  true  heading  channel  from  the  UTM  coordinates 

4.  Edit  and  filter  pitch  and  roll  channels. 

5.  Compute  corrected  UTM  coordinates  according  to  heading,  pitch,  and  roll. 

6.  Compute  detector  gate  values  for  each  of  the  21  receiver  channels.  The  detector  gate 
value  is  the  value  of  the  recorded  transient  integrated  optionally  with  an  exponential 
weighting  function  over  a  selected  time  interval. 

7.  Compute  transmitted  current  by  gating  (windowing)  the  transmitted  current  transient. 

8.  Normalize  detector  gate  values  by  transmitter  current. 

9.  Select  background  and  remove  background  (leveling). 

10.  Generate  vector  magnitude  channels  for  each  of  7  tri-axial  receiver  cubes 

1 1 .  Split  each  profile  into  7  separate  profiles,  corrected  for  heading  and  offset  distance  from 
the  platform  measure  point  (generates  7  parallel  profiles  with  13cm  offsets). 

12.  Grid  the  resulting  amplitude  data. 

13.  Apply  grid  smoothing  filters  if  necessary. 

Steps  1-9  in  the  above  numeration  constitute  preprocessing  steps  that  result  in  data  that  can  be 
exported  for  delivery  to  the  Program  Office  and/or  to  analysis  demonstrators. 

6.1.2.  Static  Data 

As  we  stated  previously,  raw  static  data  files  are  input  directly  into  MM/RMP  together  with  an 
appropriate  background  file  and  a  file  containing  RMS  noise  estimates  for  each  channel  and  time 
gate.  What  we  have  heretofore  termed  “preprocessing”  (e.g.  location  correction,  background 
removal,  etc)  is  performed  within  MM/RMP  during  the  parameter  extraction  process. 
Preprocessed  data  were  only  required  during  the  SLO  demonstration.  During  that  demonstration, 
there  were  several  organizations  that  needed  these  data  to  independently  prepare  discrimination 
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target  lists.  We  modified  MM/RMP  so  that  it  can  output  preprocessed  static  data  files  in  comma 
separated  values  (CSV)  format  as  specified  by  the  program  office.  The  data  files  were  delivered 
to  the  program  office  and  subsequently  distributed  to  other  demonstrators. 

6.2.  TARGET  SELECTION  FOR  DETECTION 

Our  target  procedure  for  target  selection  has  evolved  over  the  course  of  the  three  demonstrations 
covered  in  this  report.  We  highlight  the  evolution  through  examples. 

6.2.1.  Detection  at  YPG 


10  11  12 


10  11  12 


A)  Avg5IM1  -  YPG  Blind  Grid  B)  sPMt  Cube  M1  -  YPG  Blind  Grid 


MetalMapper  -  YPG  Blind  Grid  (Target  F1 1 ) 


At  YPG,  we  only  surveyed  the  Blind  Grid.  In  the  Blind  Grid  at  both  YPG  and  APG,  we  know 
exactly  where  to  look  for  a  target  since  whenever  a  target  is  present  it  has  been  seeded  at  the 
nominal  center  of  a  2m  x  2m  grid  square.  The  detection  maps  for  the  Blind  Grid  at  YPG  are  in 
Figure  3.1.  The  left  panel  of  that 
figure  is  the  detection  map  that 
results  when  the  magnitude  of  the 
field  (averaged  over  a  900 ps 
detection  gate)  is  compiled  into  a 
map.  The  map  has  a  spatial 
resolution  similar  to  that  of  a 
standard  EM61.  The  maps  in 
Figure  3.1  are  based  on  N-S 
profile  measurements  on  lm 
centers  and  every  other  profile 
traverses  over  the  center  points  of 
the  grid  squares.  The  2nd  panel  in 
Figure  3.1  represents  what  we  call 
the  “split-cube”  map  wherein  we 
split  out  the  response  of  each  cube 
into  independent  profiles.  Thus  1 
MetalMapper  profile  generates  a 
set  of  7  independent  profiles  on 
13cm  centers.  One  can  get  a  sense 
of  the  improved  resolution  that 
results  from  the  split  cube  map  by 
comparing  the  two  panels.  In 
Figure  6.1  we  illustrate  how  the 
split  cube  map  (6. IB)  or  profiles  can  be  helpful  in  detecting  a  target  that  is  not  manifested  as  a 
true  peak  in  the  EM61-like  Avg5IMl  detector  (6.1  A).  At  YPG  all  targets  were  picked  manually 
from  profiles  using  DIGITPROFOPT.GX  available  in  Oasis  Montaj  with  the  standard  license. 
Dynamic  data  noise  thresholds  are  well  below  0.1  pT/s. 


7  Ml  Cube  Profiles 


C)  Profile  segment  along  11  (YPG  Blind  Grid) 

Figure  6.1:  A  small  area  from  each  (A  &  B)  of  the 
maps  in  Figure  3.1  illustrating  improved  resolution  for 
target  detection  arising  from  split  cube  processing.  The 
profile  view  (C)  is  also  very  useful  finding  smaller 
and/or  deeper  targets  that  often  distort  the  side  lobes  of  a 
bigger  anomaly. 


We  benefitted  substantially  from  the  scoring  record  for  the  AOL2  system  (essentially  identical  to 
the  MetalMapper)  for  a  similar  survey  over  the  Blind  Grid  at  YPG  in  2007.  In  that 
demonstration,  we  achieved  an  overall  detection  score  of  95%  with  most  of  the  undetected 
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targets  arising  from  those  buried  at  depths  greater  than  lm.13  Upon  revisiting  YPG  with  the 
MetalMapper,  we  carefully  examined  the  dynamic  profiles  over  the  center  of  every  grid  center  to 
make  sure  that  we  picked  even  the  slightest  low-level  distortion  if  it  appeared.  As  a  result,  we 
had  a  number  of  target  picks  with  response  stage  amplitudes  on  the  range  0.1<Response<l  nT/s. 
Although  we  do  not  know  which  of  our  low-level  response  stage  picks  were  actually  detections, 
our  overall  detection  score  improved  from  95%  to  100%  and  our  score  for  deep  targets  (>lm) 
improved  from  70%  to  85%  . 

6.2.2.  Detection  at  APG 

At  APG,  we  surveyed  the  blind  grid  with  0.75m  profile  offsets  rather  than  with  lm  centers  as  we 
did  at  YPG.  This  change  was  needed  because  many  of  the  plastic  disks  used  to  mark  the  comers 
of  the  grid  squares  in  both  the  Calibration  and  Blind  grids  were  not  visible  (at  YPG,  these  “Red” 
disks  were  mostly  visible  and  provided  excellent  navigation  cues).  The  Blind  Grid  detection 
map  at  APG  is  shown  in  Figure  5.2.  Using  the  grid  centers  as  a  guide,  we  manually  examined 
the  composite  profile  “Avg5Ml”  (see  Figure  6. 1C)  and  the  7  Ml  profiles  (see  Figure  6. 1C).  If 
we  saw  a  peak  or  distortion  on  the  side  of  a  much  larger  anomaly  as  in  Figure  6. 1C  we  assumed 
the  presence  of  a  target.  As  with  the  YPG  surveys,  the  noise  level  was  very  low  («0.1  nT/s)  all 
targets  were  well  above  that  noise  level.  Our  detection  score  for  the  Blind  Grid  was  perfect  [16]. 

The  Direct  Fire  (DF)  and  Indirect  Fire  (IF)  areas  at  APG  are  in  the  Open  Field  and  closely 
simulate  real  target  areas  used  for  munition  types  seeded.  Each  of  these  areas  was  surveyed  with 
the  MetalMapper  using  parallel  profiles  on  0.75m  centers.  With  the  exception  of  an  E-W 
trending  drainage  ditch  in  the  DF,  neither  of  these  areas  had  any  real  topographic  features  to 
contend  with.  However,  both  areas  had  standing  water  in  some  areas  that  we  had  to  contend 
with.  As  a  result,  there  are  some  places  where  the  line  spacing  is  not  as  uniform  as  we  would 
have  liked.  Detection  maps  for  the  two  areas  are  shown  in  Figure  6.2  and  Figure  6.3.  In  both 
cases,  we  manually  digitized  peaks  using  Oasis  Montaj  and  DIGITPROFOPT.GX.  We 
examined  the  profiles  for  questionable  peaks.  A  strict  amplitude  threshold  was  not  used.  Rather, 
we  examined  the  profiles  on  all  low-level  target  prospects  and  if  we  had  6  or  7  cubes  responding, 
we  picked  the  peak. 

6.2.3.  Detection  at  Camp  SLO 

Target  detection  at  SLO  was  managed  in  a  completely  different  manner  than  we  have  described 
for  the  previous  2  demonstrations.  Guidance  from  the  ESTCP  demonstration  plan  for  the  SLO 
Classification  Study  required  that  all  targets  be  picked  on  the  basis  of  anomaly  threshold.  The 
applicable  detection  threshold  was  to  be  determined  by  studying  the  predicted  and  measured 
response  for  the  4  principal  targets  of  interest  (TOI)  at  SLO,  which  are  the  60mm/81mm/4.2-in 
mortars  plus  the  2.36-in  (bazooka)  rocket.  We  were  asked  to  set  our  detection  threshold  based 
on  the  predicted  response  of  the  smallest  of  the  4  TOI  when  placed  in  its  least  favorable 
orientation  (horizontal)  when  buried  at  a  depth  of  interest  of  45cm  below  ground. 


13  Our  ATC  score  was  subsequently  confirmed  independently  by  IDA  with  the  comment  that  all 
undetected  ordnance  items  were  deeper  than  1 IX. 
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Figure  6.2:  Detection  map  for  the  Indirect  Fire  area  of  the  APG  standardized  test  site.  The 
parameter  mapped  is  the  average  of  vertical  secondary  field  for  the  5  inner-most  cubes  of  the 
MetalMapper  receiver  array.  The  “x”  symbols  mark  a  target  pick. 
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Figure  6.3:  Detection  map  for  the  Direct  Fire  area  of  the  APG  standardized  test  site.  The 
parameter  mapped  is  the  average  of  vertical  secondary  field  for  the  5  inner-most  cubes  of  the 
MetalMapper  receiver  array.  The  “x”  symbols  mark  a  target  pick. 
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Figure  6.4:  Base  map  of  the  1 1.8-acre  study  site  at  Camp  San  Luis  Obispo.  The  map  shows 
the  subdivision  of  the  area  into  54  survey  blocks  (30m  x  30m),  and  1 1  sub-areas.  Also  shown 
are  the  target  picks  based  on  the  MM  dynamic  survey.  Picks  marked  in  green  comprise  the 
picks  for  which  ground-truth  was  supplied  to  all  data  processing  demonstrators.  The  blue 
picks  represent  targets  for  which  demonstrators  submitted  a  prioritized  dig  list.  The  8  blocks 
in  the  rectangular  area  K6-N7  plus  the  narrow  strip  outlined  in  orange  are  areas  that  were 
excluded  from  the  study  for  budgetary  reasons. 
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We  generally  followed  the  methods  and  rationale  developed  by  Nelson,  et.  al.  [17]  in 
determining  the  threshold  to  apply  for  target  detection  at  SLO.  First,  we  computed  average 
polarizability  transients  for  the  4  targets  of  interest.  We  computed  these  averages  using  static 
data  sets  that  we  measured  with  the  MetalMapper  over  the  4  targets  of  interest.  We  were  able  to 
use  data  from  YPG  and  APG  for  the  60mm  and  81mm  mortars.  We  augmented  those  data  with 
static  free-air  measurements  of  the  4  TOI  available  to  demonstrators  at  SLO  for  use  in  the  test  pit 
and  with  static  measurements  over  other  targets  buried  in  the  test  strip.  We  ended  up  with 
approximately  15  data  sets  for  each  target  type  representing  a  range  of  depths  and  orientations 
with  respect  to  the  antenna  platform.  The  results  are  shown  in  Figure  6.5.  From  these  curves, 
we  are  able  to  estimate  the  gated  principal  polarizability  values  (i.e.  the  “betas”)  for  the  time  gate 
(1 00<t<900ps)  that  we  used  in  processing  the  dynamic  data  to  generate  a  detection  parameter. 
These  “betas”  were  used  in  modeling  the  MetalMapper  response  to  each  of  the  4  targets  for  a 
range  of  depths  when  positioned  directly  beneath  the  antenna  platform.  Two  target  attitudes 
were  modeled:  vertical,  and  horizontal.  The  results  of  the  modeling  exercise  for  the  4  targets  for 
the  case  of  “split  cube”  processing  is  shown  in  Figure  6.6.  The  Figure  suggests  that  the  smallest 
detection  threshold  will  be  about  3  nT/s  for  the  horizontal  2.36-in  rocket  at  a  depth  of  0.66m 
below  the  platform  (0.45cm  below  ground  level). 


60mm  Mortar  —  SLO  Static  &  Test  Strip 


Figure  6.5:  The  average  principal  polarizability  curves  (red, green, blue)  for  the  4  targets  of 
principal  interest  at  Camp  SLO.  These  curves  were  derived  from  approximately  15 
independent  static  MetalMapper  measurements  acquired  in  the  free-air  (SLO  Test  Pit),  and  in 
the  ground  (SLO  Test  Strip  ).  The  average  curves  were  used  in  estimating  the  target  detection 
threshold  to  be  used  at  SLO. 
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60mm(IF):  Detection  Response  (SplitCube  -  Z)  vs  Depth 


81mm(IF):  Detection  Response  (SplitCube  -  Z)  vs  Depth 


Depth  (m)  Depth  (m) 

Figure  6.6:  Detection  plots  for  the  4  targets  of  interest  at  SLO.  The  curves  represent  the 
peak  response  as  a  function  of  depth  below  the  antenna  platform  for  two  target  attitudes: 
vertical,  and  horizontal.  The  MM  platform  when  deployed  on  skids  is  2 1  cm  above  ground 
level.  The  dashed  vertical  green  line  has  been  placed  at  a  depth  of  66cm  (45cm  below  ground 
level).  The  noise  floor  (shown  correctly  for  the  60mm  and  2.36-in  targets  is  around  0. 1  nT/s. 


During  our  dynamic  survey  at  SLO,  we  surveyed  the  test  strip  a  minimum  of  twice  a  day.  Each 
survey  of  the  test  strip  calls  for  acquiring  data  along  the  test  strip  in  reciprocal  directions.  As  a 
result,  we  acquired  a  total  of  60  profiles  over  the  test  strip,  30  in  each  direction.  We  used  these 
data  to  experimentally  verify  the  modeling  results  shown  in  Figure  6.6.  So  that  the  peak 
amplitudes  mimic  so  far  as  possible  the  same  amplitudes  that  we  would  measure  during  a 
dynamic  survey  over  an  area,  we  processed  the  repeated  test  strip  profiles  as  if  they  were  an  area. 
We  did  that  follows: 


1 .  Perform  steps  1  &  2  as  specified  in  section  6.1.1. 

2.  Translate  coordinate  system  so  that  the  origin  is  located  at  target  T-001 
(705417.00E,3913682.00N). 

3.  Rotate  coordinates  clockwise  through  an  angle  of  32.4712°.  This  aligns  the  local  Y  axis 
with  the  direction  of  a  line  extending  from  target  T-001  to  T-010  of  the  test  strip. 

4.  Offset  each  profile  in  the  local  X  direction  by  a  distance  (n-l)*lm  where  n  is  the  profile 
number.  This  creates  a  series  of  parallel  profiles  offset  from  its  neighbor  by  a  nominal 
offset  of  lm. 

5.  Continue  the  normal  sequence  of  dynamic  processing  (step  numbers  3-13;  section  6.1.1) 
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The  modified  processing  steps  produce  a  “pseudo-map”  of  the  calibration  data  as  we 
illustrate  in  Figure  6.7.  We  completed  the  processing  by  running  a  threshold  peak  picker  and 
then  editing  out  the  spurious  peaks.  The  resulting  target  list  is  easily  sorted  by  rows  and 
allowed  us  to  measure  the  mean  and  standard  deviation  of  the  peaks. 

The  astute  observer  will  notice  that  the  pseudo-map  shown  Figure  6.7  can  be  divided  into  3 
vertical  zones  (A,B,&C).  The  basis  for  this  sub-division  is  the  fact  that  we  changed  from 
surveying  on  skids  to  using  wheels  and  then  back  to  skids  at  the  end  of  the  survey.  We  used 
the  wheel  deployment  during  the  roughest  part  of  the  survey.  The  wheel  deployment  raises 
the  MetalMapper  antenna  platform  an  additional  8.5cm  above  the  ground  level  and  the  result 
of  that  higher  platform  elevation  produces  a  noticeable  change  in  the  texture  of  the  map.  We 
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Figure  6.7:  A  “Pseudo”  split  cube  map  consisting  of  60  calibration 
profiles  acquired  over  a  period  of  approximately  3  weeks.  Each  profile 
has  been  offset  from  its  neighbor  by  lm.  There  are  10  calibration 
targets  (rows)  at  approximately  6m  intervals  starting  a  0m.  There  are 
also  several  spurious  “uninvited”  targets  the  most  obvious  of  which  is 
the  one  that  forms  the  row  at  about  30m. 
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computed  the  statistics  for  each  of  the  10 
targets  in  the  test  strip  for  each  of  the 
configurations.  When  we  adjust  for 
platform  height,  the  target  depths  with 
respect  to  the  platform  are  51cm  (skid 
deployment)  and  60cm  (wheel 
deployment).  In  Figure  6.8,  we  have 
plotted  the  mean  target  amplitudes  as 
obtained  from  the  repeated  test  strip 
profiles  for  each  of  deployment 
configurations.  The  plotted  experimental 
points  are  very  close  to  the  model  curve  for 
the  2.36-in  rocket.  The  measured  responses 
are  a  bit  low  for  the  mortar.  In  any  case, 
these  results  suggest  that  a  threshold  of  3 
pT/s  is  appropriate  for  the  MetalMapper 
when  deployed  on  skids  while  the  threshold 
must  be  lowered  to  2  pT/s  when  it  is 
deployed  with  wheels.  We  used  thresholds 
of  3  and  2  for  detecting  peaks. 


60mm(IF):  Detection  Response  (SplitCube  -  Z)  vs  Depth 
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Figure  6.8:  Annotated  detection  figures  for 
the  2.36-in  rocket  and  the  60mm  mortar. 

The  plotted  points  are  average  peak 
amplitudes  for  the  vertical  (‘x”)  and 
horizontal  (‘+’)  buried  targets. 
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6.3.  PARAMETER  ESTIMATES 


The  conceptual  processing  flow  is  illustrated  in  Figure  6.9.  The  figure  shows  the  63  transients 
comprising  a  data  set  on  the  left.  The  panel  on  the  right  are  plotted  the  three  principal 
polarizability  rate  transients.  We  call  the  computer  program  that  accomplishes  this 
transformation  MM/RMP.  The  basis  of  MM/RMP  is  the  ubiquitous  point  dipole  model 
approximation  for  approximating  the  EMI  response  of  a  small  conducting  and  magnetically 
permeable  object.  Byproducts  of  this  modeling  process  are  estimates  of  the  target  position  and 


Principal  Polarizability  Transients 
(“Intrinsic”  target  parameters) 


60imn  Mortar 


time  (ins) 


Tgt  Position/Attitude 
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Figure  6.9:  Parameter  extraction  using  MM/RMP.  The  input  data  are  transients  (63) 
acquired  during  a  single  static-mode  measurement  with  the  MetalMapper.  The  outputs  are 
the  principal  polarizability  rate  transients,  target  position,  and  target  attitude. 


attitude  angles.  The  dipole  model  approximation  is  widely  used  within  the  UXO  community  for 
interpreting  UXO  [19-23],  A  particularly  thorough  and  systematic  treatment  of  the  theory  can  be 
found  in  chapter  2  of  Leonard  Pasion’s  PhD  thesis  [24],  The  algorithm  used  in  MM/RMP  was 
developed  by  Torquil  Smith  at  Lawrence  Berkeley  Labs  (LBL)  [25],  The  basic  theory  for  the 
dipole  approximation  of  EMI  response  for  small  objects  is  summarized  in  Figure  6.10.  We 
outline  the  essential  elements  of  the  theory  below: 


1.  We  assume  that  the  target  is  dimensionally  small  as  compared  with  the  distances  between 
the  target  and  the  transmitter  loop(s)  and  the  receiver  loop(s).  We  also  assume  that  the 
conductivity  of  the  host  medium  is  low  compared  with  the  conductivity  of  the  target 
object  and  therefore  can  be  neglected  (i.e.,  “free-space”  approximation). 
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2.  To  the  extent  that  the  assumptions 
in  1  are  valid,  we  can  approximate 
the  behavior  of  the  target  as  a 
point  dipole  characterized  with  a 
time-  or  frequency  varying 
polarizability  tensor  (P). 

3.  The  primary  magnetic  field  (Hp) 

“illuminates”  the  target  object, 
which  is  assumed  to  be  very  highly 
conducting  and  (possibly) 
permeable.  In  Figure  6.10,  the 
primary  field  is  generated  by 
transmitter  loop  (red)  carrying  a 
time  varying  current. 

4.  Time  variations  in  the  primary 
field  induce  eddy  currents  to  flow 
in  the  target.  At  any  particular 
time  or  frequency,  the  magnitude 
and  direction  of  the  resulting 
dipole  moment  is  approximated  as 
the  tensor  product  of  the  primary 
field  vector  with  the  polarizability 
tensor  ( M  =  P  •  IF ). 

5.  Given  the  dipole  moment  vector  M  at  any  particular  time  or  frequency,  the  value  of  the 
secondary  field  at  the  receiver  loop  is  determined  by  computing  the  tensor  product 
between  well-known  dipole  Green’s  functions  and  the  moment  (  B  =  /r0G  ■  M  ) . 

6.  As  suggested  in  Figure  6.10,  the  polarizability  tensor  P  is  a  2nd  rank  symmetric  tensor 
having  6  independent  elements.  In  such  cases,  there  exists  a  coordinate  system  called  the 
“principal  axes”  coordinate  system  in  which  the  polarizability  tensor  is  diagonal  with 
only  3  elements.  The  “principal  axes”  coordinate  system  and  the  observation  coordinate 
system  are  related  by  means  of  an  orthogonal  transformation  (T).  The  orthogonal 
transformation  can  be  expressed  in  terms  of  3  elementary  rotation  angles  (cp,0,\|/) 
sometimes  called  the  “Euler  Angles”.  At  any  particular  time  or  frequency,  the  response 
at  the  receiver  is  a  function  of  9  parameters:  1)  the  3  coordinates  of  the  target  position,  2) 
the  3  Euler  Angles,  and  3)  the  3  diagonal  elements  of  the  polarizability  tensor  evaluated 
at  a  particular  time  or  frequency. 

7.  Given  sufficient  independent  measurements  of  the  secondary  field  B  one  can  estimate  the 
9  unknown  parameters  enumerated  in  6  using  well-known  principles  of  non-linear  and/or 
linear  inversion.  It  is  beyond  the  scope  of  this  report  to  delve  into  the  details  of  that 
process.  One  can  view  the  inversion  process  as  simply  removing  the  effect  of  spatial 
dispersion  produced  when  the  source  dipole  (M)  is  convolved  with  the  dipole  Green’s 
function  G.  Such  a  “deconvolution”  results  in  an  estimate  of  the  dipole  moment  vector 
(M).  M  is  the  tensor  product  between  the  primary  field  Hp  and  the  unknown 
polarizability  tensor  (P).  Thus  finding  the  6  unknown  components  of  the  polarizability 
tensor  becomes  a  linear  problem  if  the  position  of  the  target  is  known.  Likewise,  spatial 
“deconvolution”  requires  knowledge  of  target  position  so  that  the  Green’s  function  G  can 


B(f-r0,r0-f';O  =  HG(r-f0){^(O'H'(fc-r')} 
r  =  Rx  position;  r  =  Tx  position;  r0  =  target  position 
G  (  r  -  r  )  =  Dipole  Green's  functi  on 
P  (t )  =  Polarizability  rate  tensor  (symmetric  w/  6  unknowns) 
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H?  (r0  -  r  )  =  Primary  magnetic  field  intensity  (function  of  r0). 

T  ( 3, if/)  =  Orthogonal  transformati on  (3  unknowns ) 
p[(t),p'2  (0  =  Principal  polarizability  transients 

Figure  6.10:  Summary  of  the  theory  for 
approximating  the  response  of  a  small 
conducting  and  permeable  object  with  a  point 
dipole. 
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be  computed.  Thus,  the  essential  problem  in  extracting  parameters  ultimately  reduces  to 
estimating  the  3  unknown  coordinates  of  target  position. 


6.3.1.  Implementation  Issues 

Figure  6.11  is  a  very  much  simplified  flow  diagram  of  the  inversion  process  as  it  has  been 
implemented  in  MM/RMP.  In  the  first  step,  an  optimal  position  is  estimated  (a  non-linear 
problem)  and  then  for  each  time  gate  the  6  independent  elements  of  the  polarization  tensor  (in 
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Figure  6.11:  Flow  diagram  showing  the  how  dipole  modeling  is  implemented  in  MM/RMP. 
The  problem  is  divided  into  two  estimation  problems,  one  non-linear,  the  other  linear.  These 
problems  are  solved  for  each  time  gate  in  the  MetalMapper  data. 


the  observation  coordinate  system)  are  estimated.  The  latter  problem  is  a  series  of  N  linear 
estimation  problems  with  N  being  the  number  of  time  gates  analyzed.  And  finally,  the  last  step 
is  the  estimation  of  the  principal  coordinate  system.  The  results  of  the  last  step  provide  estimates 
of  the  polarizability  rate  at  each  time  gate  plus  the  Euler  Angles  specifying  the  relationship 
between  the  observation  coordinate  system  and  the  principal  coordinate  system  (a  target-fixed 
coordinate  system).  This  problem  is  linear.  However,  there  are  at  least  two  ways  to  undertake  it: 


1.  Time-Varying  Attitude  Angles  (Type  0):  In  this  case,  attitude  angles  are  computed  at 
each  time  gate.  This  method  permits  the  attitude  to  vary  with  time.  Usually,  the 
electromagnetic  attitude  remains  the  same  over  time.  However,  there  are  certain  target 
types  where  the  apparent  attitude  changes  with  time  (Figure  6.12). 
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2.  Fixed  (Average)  Attitude  Angles  (Type  1):  In  this  case,  we  assume  that  the  target 
attitude  is  a  constant  in  time  and  can  be  determined  by  computing  the  Euler  Angles  based 
on  a  time-averaged  polarization  tensor.  Using  these  angles,  the  polarization  tensor  at 
each  time  gate  is  transformed  into  the  principal  coordinate  system.  The  diagonal 
elements  of  these  tensors  are  assumed  to  be  the  values  of  the  principal  polarizability 
transients.14 


Figure  6.12:  Two  examples  from  the  YPG  Calibration  Grid  illustrating 
the  difference  between  Type  0  and  Type  1  tracking  of  the  principal 
polarizability  curves.  This  behavior  only  occurs  with  targets  exhibiting  a 
cross-over  in  the  curves  suggesting  that  the  targets  apparent  attitude 
changes  with  time. 


14  By  convention,  we  order  the  principal  polarizabilities  in  descending  order.  This  means  that  the 
axis  of  maximum  principal  polarizability  is  the  target  x-axis.  For  UXO-like  bodies,  the  axis  of 
symmetry  will  always  correspond  to  the  x-axis. 
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In  most  UXO  types,  the  principal  polarizability  curves  for  the  two  processing  methods  come  out 
remarkably  alike.  And  in  these  cases,  the  Type  1  method  seems  to  produce  better  polarizability 
curves.  That  is  to  say  that  the  electromagnetic  attitude  remains  roughly  constant  with  time. 
However,  there  are  cases  where  the  attitude  angles  change  in  time  and  the  two  different  methods 
(i.e.,  Type  0  and  Typel)  produce  different  results.  Two  examples  from  YPG  that  illustrate  the 
problem  are  shown  in  Figure  6. 12. 15  These  curves  were  extracted  from  a  155mm  M183A1  (6.12- 
top)  projectile  and  a  BDU-28  submunition.  In  the  figures,  the  plotted  symbols  represent  the 
principal  polarizability  values  estimated  at  each  time  gate.  The  solid  lines  represent  the  fit  of  a 
4-term  parametric  relation  to  each  of  the  3  principal  polarizability  transients.  The  transients  have 
been  colored  according  to  the  convention  that  red  is  the  major  (i.e.  largest)  curve  and 
corresponds  to  the  x-axis  of  the  principal  coordinate  system.  The  Green  and  Blue  curves  are  in 
decreasing  size  and  are,  respectively,  the  curves  for  the  y-axis  and  z-axis.  In  the  Type  1  curves 
on  the  left,  note  how  points  of  a  particular  color  (e.g.,  red)  track  through  the  cross-over  point  and 
link  logically  with  points  of  the  same  color  on  the  opposite  side  of  the  cross-over.  Now  look  at 
the  corresponding  Type  0  curves  (right)  for  the  same  target.  In  both  examples,  the  blue  points  to 
the  left  of  the  cross-over  point  appear  to  link  logically  with  red  points  to  the  right.  This  is  an 
artifact  caused  by  the  fact  that  Type  0  curves  are  produced  by  computing  the  principal  coordinate 
system  independently  for  each  time  gate.  Part  of  the  process  of  computing  the  coordinates  is  to 
sort  the  resulting  eigenvalues  (i.e.,  principal  polarizability  elements)  in  decreasing  order.  As  we 
pass  a  cross-over  therefore  the  principal  polarizability  values  are  sorted  into  a  different  order  so 
that  the  largest  principal  polarizability  remains  with  the  x-axis  of  the  principal  coordinate  system. 
This  does  not  happen  in  the  case  of  Type  1  curves  because  the  principal  coordinate  system  is 
only  computed  once  using  average  principal  polarizabilities.  Our  discrimination  techniques  are 
all  based  on  Type  1  polarizability  curves  or  from  meta-parameters  derived  from  them. 

6.3.2.  Meta-Parameters 

Parametric  Curve  Fits  -The  three  principal  polarizability  curves  such  as  those  shown  in  Figure 
6.12  contain  all  the  information  available  about  the  target.  For  the  three  demonstrations 
discussed  in  this  report,  each  curve  consists  of  42  points  spaced  logarithmically  in  time  over  the 
range  105<t<7912ps.  Many  authors  have  noted  that  the  principal  polarizability  transients  are 
often  well  described  by  parametric  functions  of  time  or  frequency  as  appropriate  with  as  few  as  3 
or  4  parameters  [26-28].  As  a  standard  operation  during  the  processing  of  data  through 
MM/RMP,  we  parameterize  each  curve  using  a  form  of  the  parametric  relation  proposed  by 
Smith  [28].  It  has  the  form 


dP(t)  _  ,  (  ,  P  \ 
dt  ky  2 (Vto  +  t)J 


(6.1) 


The  equation  (6.1)  has  4  parameters  (k,a,/3,y  ).  Each  of  these  parameters  has  a  physical 
interpretation. 


15  An  error  in  the  plotting  routine  incorrectly  labels  the  dual  polarizability  plots  in  Figure  6.12. 
We  have  corrected  the  labels.  However,  Type  1  plots  appear  on  the  left  rather  than  on  the  right. 
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1.  k  -  this  parameter  has  units  of  cm3  when  time  is  expressed  in  units  of  ps  and  dP/dt  is  in 

'J  O 

units  of  cm  /ps  (equivalent  to  m  /s).  This  parameter  approximates  the  polarizability  at 
the  so-called  “inductive  limit”  ( Po=P(t=0 )). 

2.  a  -  Smith  [28]  relates  this  parameter  to  transition  zone  between  the  early  time  and 
intermediate  time.  With  MetalMapper  data,  this  parameter  is  not  very  useful.  Typically 
our  fits  estimate  values  for  a  in  the  range  l<a<100ps.  Although  we  sometimes  measure 
time  gates  earlier  than  100  ps  we  have  found  with  MetalMapper  data  that  these  early  time 
data  are  overwhelmed  by  the  amplitude  of  the  background  signal.  Consequently,  we  start 
our  analysis  of  transient  decays  at  100  ps. 

3.  /?  -  This  is  the  intermediate  power-law  decay  rate.  Typically  it  takes  values  in  the  range 
0.5<P<2.5. 

4.  y  —  This  is  the  principal  late-stage  decay  rate.  The  transient  fields  arising  from  eddy 
currents  in  a  conducting  body  all  degenerate  at  late  time  to  a  single  exponential 
decay[29].  For  many  targets,  however,  the  amplitude  of  the  signal  at  late  time  is  simply 
too  low  to  see  the  late-stage  decay  much  less  to  estimate  it. 

We  have  found  that  our  efforts  to  parameterize  the  principal  polarizability  transients  by  fitting 
them  to  parametric  relations  such  as  6.1  often  fail.  Probably  this  is  due  to  a  lack  of  sophistication 
on  our  part  with  regard  to  robust  fitting  since  others  seem  to  have  had  success  in  such 
parameterization.  In  any  case,  we  have  not  used  these  parameters  in  our  efforts  to  discriminate 
UXO  targets.  Instead,  we  use  the  transients  directly. 

Scalar  Meta-Parameters  -  Many  have  found  success  with  target  discrimination  using  scalar 
parameters  derived  from  much  simpler  instruments.  Moments  of  the  principal  polarizability 
transients  are  easy  to  compute  numerically  and  very  robust  with  respect  to  noise.  We  have 
benefitted  from  the  insight  on  the  relationship  of  moments  to  meaningful  target  features  provided 
in  a  paper  by  Smith  and  Jones  [30].  Mathematically,  the  nth  moment  of  the  polarizability  is 
defined  as 

Pn  - 1  tn  P{t)  dt ;  where  P(t)  is  a  polarizability  rate  transient.  (6.2) 

We  calculate  the  0th  and  1th  moments  for  each  of  the  3  principal  polarizability  transients  thereby 
producing  6  scalar  “meta-parameters”  (Pflx,  Poy,  Poz,  Pix,  Piy,  Piz )  where  the  subscripts  (x, y,  and  z) 
denote  the  principal  axes  coordinates. 

For  each  of  these  scalar  parameter  sets  (i.e.  Po,Pi )  we  calculate  four  additional  scalar  values  two 
of  which  relate  to  target  shape: 

Ip.  p . 

1.  Transverse  Target  Polarizability:  PiT  =-  -  - ;  t  =  0,1  -  This  parameter  is  the 

Pix 

normalized  geometric  mean  of  the  polarizability  moments  transverse  to  the  axis  of 
maximum  polarizability  moment. 

P 

2.  Target  Aspect:  />=-£.;  i=Ql  -  This  is  a  measure  of  the  aspect  or  elongation  of  the 

lR  p  9  9 

r IT 

target.  Values  of  PiR>  1  suggest  rod-like  targets  while  PiR<  1  suggest  flat  targets 


ESTCP  Final  Report  MM-0603: 


48 


MetalMapper 


3. 


4. 


(Piy-Piz) 

Target  Eccentricity:  PiE  =  — - — ;  i  =  0,1  -  This  is  a  measure  of  the  eccentricity  of 

^ix 

the  target.  A  target  with  good  symmetry  with  respect  to  the  principal  x  axis  will  have 

P  ()E<<  I  • 


Target  Time  Constant: 


-  This  parameter  has  the  dimensions  of  time. 


If  the 


transient  involved  were  a  true  exponential  decay,  then  the  ratio  of  the  first  and  second 
moments  can  be  shown  to  be  equal  to  the  time  constant.  For  UXO  targets  this  will  never 
be  the  case.  However,  we  can  view  the  parameter  as  a  measure  of  the  time  persistence  of 
the  transient  with  higher  values  meaning  longer  decays.  For  the  MetalMapper 
demonstrations  discussed  here,  the  time  constant  values  maximize  at  around  5000  ps  for 
large  munitions  (e.g.,  81mm,  105mm,  4.2-in,  etc).  Time  constants  values  for  clutter  and 
munitions  debris  are  typically  in  the  range  x<2000  ps. 
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6.4.  CLASSIFIER  AND  TRAINING 


Our  efforts  to  develop  a  process  for  discrimination  reflects  an  evolution  over  the  course  of 
demonstrations  at  YPG,  then  at  APG,  and  finally  at  San  Luis  Obispo.  We  believe  that  the  ATC 
and  IDA  scoring  records  reflect  the  fact  that  with  each  demonstration  we  have  improved  not  only 
on  our  scores  but  also  in  our  ability  to  effectively  discriminate. 

Discrimination  starts  with  the  estimation  of  I  ' 

1  1  p  1  Aberdeen2Stat00047-Rrincipal  Polarizability  Curves  (Type  1) 

target-related  parameters  from  our  data. 

As  we  have  recounted  in  section  6.3,  we 
approximate  the  target  with  a  point  dipole- 
model  which  results  in  the  extraction  of  the 
principal  polarizability  curves  (see  Figure 
6.9).  Our  view  of  a  UXO  and  the  behavior 
of  its  principal  polarizability  transients 
focus  (perhaps  too  narrowly)  on  shape. 

When  viewed  in  the  principal  coordinate 
system,  the  polarizability  of  a  UXO  will 
manifest  itself  by  exhibiting  a  single  large 
(major)  polarizability  transient  over  two 
smaller  (minor)  transients.  Such  behavior 
is  illustrated  in  Figure  6.5,  Figure  6.9,  and 
Figure  6.12.  Likewise,  clutter-like  objects 
do  not  exhibit  such  symmetry  properties 
(see  Figure  6.13). 

When  we  look  at  scalar  parameters,  we  see 
much  the  same  thing.  Figure  6.14  is  a 
categorized  scatter  plot  using  scalar  meta 
parameters  defined  in  sections  6.3.  The 
static  data  are  from  the  Calibration  Grid  at 
APG.  All  data  points  plot  above  the  line 
P ox=P ot  suggesting  that  all  targets  are 
elongate.  The  BLU-26,  BDU-28, 

Calibra  Ball,  and  105mm(DF)  plot  very 
close  to  the  line  suggesting  that  they  are 
equi-dimensional  (sphere-like).  Each 
munition  type  clusters  together  on  the  plot. 

However,  the  points  representing  clutter 
are  scattered  over  a  broad  range. 

In  theory,  the  0th  moment  parameter  (i.e., 

Pot  has  the  units  of  volume  and  is  related  to 
target  size.  We  do  not  have  volumes  for  the  targets  at  YPG  and  APG,  but  we  do  have  their 
weights.  In  Figure  6.15,  we  have  plotted  the  RMS(ZV)  against  the  target  weights  in  order  to 
demonstrate  the  direct  relation  between  P0  and  target  size.  We  have  color-coded  the  points 
according  to  three  categories  (small,  medium,  and  large)  that  have  been  defined  by  ATC.  This 


Aberdeen2Stat00047 -Principal  Polarizability  Curves  (Type  1) 


Figure  6.13:  Example  of  principal  polarizability 
curves  for  a  clutter  object.  The  object  does  not 
exhibit  the  symmetry  characteristic  of  UXO. 
This  object  is  buried  in  the  Calibration  Grid  (cell 
D2)  at  APG. 
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Figure  6.14:  A  categorized  scatter  plot  made 
with  two  scalar  parameters  extracted  from  data 
taken  over  the  calibration  grid  at  APG. 
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Figure  provides  us  with  excellent  empirical 
evidence  that  any  measure  of  integrated 
polarizability  can  be  directly  related  to  target 
size. 

6.4.1.  Discrimination  at  YPG 

We  have  used  several  plots  of  the  principal 
polarizability  transients  to  illustrate  various 
points  (see  Figures  5.3,  Figure  6.5,  Figure 
6.12,  and  Figure  6.13).  With  good  data  ,  an 
interpreter  can  easily  discriminate  between 
items  that  are  most  likely  targets  of  interest 
(TOI)  and  items  that  are  most  likely  clutter. 

The  visual  (and  numerical)  clues  are: 

1.  Object  Size  -  Size  as  suggested  by 
the  3  scalar  values  (Pox,  Poy,Poz)  can 
be  used  to  narrow  the  possibilities  to 
one  of  three  size  classes  (see  Figure 
6.15). 

2.  Symmetry  (Eccentricity)  -  In  most 
cases,  the  principal  polarizability 
curves  should  indicate  a  rod-like  body 
having  a  large  major  polarizability 
curve  and  2  smaller  minor  polarizability  curves  of  roughly  the  same  size  and  shape.  This 
is  the  case  for  all  examples  we  have  shown  thus  far.  It  is  important  to  be  aware,  however, 
that  the  target  may  not  always  exhibit  perfect  symmetry.16  Note  the  symmetry  or 
eccentricity  property  is  a  function  of  time. 

3.  Aspect  Ratio  -  The  ratio  of  the  major  curve  to  the  geometric  mean  of  the  minor  curves. 
This  ratio  is  a  function  of  time. 

4.  Major  Polarizability  Shape  -  The  shape  of  the  largest  principal  polarizability  curve  is 
often  very  distinctive. 

For  the  Blind  Grid  target  list  at  YPG,  we  used  the  polarizability  plots  from  each  of  the  target 
cells  to  decide  whether  the  target  was  a  “O”  (Ordnance)  or  “C”  (Clutter).  The  prerequisite  for 
that  decision  was  that  the  target  exhibit  symmetry  and  correlate  (visually)  with  a  catalog 
containing  19  “Type”  polarization  curves.  This  catalog  included  both  the  graphical  behavior  of 
the  principal  polarizabilities  for  each  of  the  19  ordnance  types,  and  the  values  of  the  various 
scalar  meta-parameters  that  are  computed  by  MM/RMP  and  stored  in  the  target  database.  As  we 
suggest  in  Figure  6.16,  the  decision  about  whether  the  target  is  ordnance  or  clutter  was  made  by 
the  interpreter.  In  making  the  decision,  the  interpreter  assigns  a  numerical  “ Grade ”  in  the  range 
(0,3)  that  indicates  a  rough  confidence  level  in  that  decision.  As  a  second  stage,  a  decision 
metric  (Mo)  in  the  range  (0,6)  is  computed  according  to  the  formulas  shown  in  Figure  6.16.  A 


16  Symmetry  properties  of  the  principal  polarizability  curves  are  sometimes  compromised  by 
large  lateral  offsets  between  the  target  and  the  center  point  of  the  antenna  array.  Symmetry  can 
also  be  compromised  by  pitch  angles  between  30°  and  60°.  A  TOI  may  have  some  physical 
damage  that  changes  its  principal  polarizability  curves. 
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Figure  6.15:  A  scatter  plot  showing  the 
correlation  between  target  weight  and  object 
size.  The  symbols  are  colored  (red, green, blue) 
to  indicate  3  categories  of  munitions  (small, 
medium,  and  large). 
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Figure  6.16:  Flow  diagram  for  discrimination  as  applied  to  detected  targets  from  the  Blind 
Grid  at  YPG  (June  2008). 


score  of  MD=6  indicates  highest  confidence  Ordnance,  while  MD=0  indicates  highest  confidence 
clutter. 


Admittedly,  this  was  a  crude  and  basically  “unprincipled”  method  for  preparing  a  target  list. 
Nonetheless,  we  achieved  a  very  respectable  discrimination  score  of  100%  (rounded  to  the 
nearest  5%)  [15]. 

6.4.2.  Discrimination  at  APG 

In  preparing  our  target  lists  for  the  APG  demonstration,  we  were  aided  considerably  by  its 
reconfiguration  completed  early  in  2008.  As  a  result  of  that  reconfiguration,  the  Blind  Grid  now 
contains  only  6  ordnance  types.  And  the  Direct  and  Indirect  Fire  Areas  each  contain  3  types, 
which  are  mutually  exclusive  subsets  of  the  6  types  seeded  in  the  Blind  Grid.  Building  on  our 
experience  at  YPG,  we  tried  to  develop  a  more  principled  approach  to  discrimination.  This 
approach  was  based  on  3  levels  of  principled  discrimination: 

1 .  Data  Quality 

2.  Rule-based  Decisions 

3.  Library  Curve  Matching 

A  schematic  flow  diagram  of  the  discrimination  process  we  used  for  discrimination  of  our  APG 
static  data  is  shown  in  Figure  6.17  below. 
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Figure  6.17:  A  flow  diagram  of  the  discrimination  process  as  it  was  applied  for  the 
discrimination  stage  of  the  APG  demonstration.  The  discrimination  decisions  were,  for  the 
most  part,  founded  on  rule-based  decisions  and  library  curve  matching  wherein  a 
multiplicative  penalty  function  based  on  data  quality  was  applied  to  the  decision  metrics. 


Data  Quality 

Our  experience  with  MM/RMP  indicates  that  if  a  static  data  set  has  an  SNR  >10  and  the  target  is 
an  isolated  object,  we  can  nearly  always  expect  the  fit  statistic  resulting  from  a  MM/RMP  model 
fit  to  the  data  to  be  greater  than  90%. 17  Figure  6.18  is  a  3D  plot  showing  the  SNR  and  the  Fit 
from  79  seeded  targets  from  the  APG  calibration  grid.  The  third  dimension  (labeled  Ma)  is  a 
“figure-of-merif  ’  (FOM)  value  derived  from  SNR  and  Fit.  We  have  also  modulated  the  color  of 
the  plot  points  with  Ma  (warm  colors  for  high  Ma\  cool  colors  for  low  Ma ).  Only  4  points  in  the 
figure  fall  off  the  trend.  Those  points  have  SNR>20dB  and  Fit  statistics  70<Fit<80%  and 
correspond  with  M42  and  BLU-26  targets,  small  submunitions  that  for  some  reason  are  not  fit 
well  with  a  dipole  model.  We  used  the  figure-of-merit  parameter  Ma  as  a  measure  of 
discrimination  for  measured  data  sets  with  regard  to  their  analyzability.  Based  on  our  study  of 
the  APG  cal  lanes,  a  threshold  value  of  Ma> 0.5  gets  all  the  good  analyzable  data. 


17  A  perfect  fit  is  100%. 
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Rule-Based  Decisions 

The  categorized  scatter-plot  shown  in  Figure  6.14 
shows  how  the  6  targets  of  interest  are  separated 
and  well  grouped  by  two  scalar  parameters  ( Pox 
and  Ppt).  The  figure  also  illustrates  that  targets 
having  values  of  Pox<rl 00  cm3  are  not  targets  of 
interest  provided  that  we  can  believe  the 
estimates  of  Pox  that  are  provided  by  MMRMP. 

We  are  able  to  make  this  statement  because  at 
APG,  there  are  only  6  targets  of  interest.  Based 
on  the  a  priori  knowledge  about  the  TOIs, 
together  with  our  knowledge  of  the  relationship 
between  SNR  and  Fit  to  the  quality  of  the  target 
parameters,  we  are  able  to  apply  a  set  of  rule- 
based  decisions  that  serve  to  classify  many  of  the 
targets.  As  suggested  Figure  6.17,  the  following 
rules  are  applied: 

1.  All  targets  with  a  value  of  Ma<0.5  (i.e., 
low  SNR  and/or  Fit)  are  designated  as 
“Cannot  Analyze.”  These  targets  must 
be  dug  in  order  to  preclude  the  possibility 
that  some  may  be  ordnance. 

2.  All  targets  with  a  value  of  Pox>  Poxmax  or  Pox<  Poxmin  will  be  designated  as  clutter  with  a 
decision  metric  value  of  Md=Ma  where  Ma  is  the  figure-of-merit  factor  indicating  data 
quality. 

Values  for  Poxmax  and  Poxmm  are  determined  by  computing  the  statistics  (mean  and  standard 
deviations)  for  the  scalar  parameters  associated  with  each  of  the  target  types  of  interest.  For  this 
purpose,  we  used  the  data  sets  we  acquired  over  the  Calibration  Grid.  In  Table  6.1  we  have 
tabulated  the  observed  mean  and  standard  deviations  for  the  6  TOI’s  together  with  the  values  for 
Poxmax  and  Poxmin ■  The  minima  were  taken  to  be  the  mean  minus  two  standard  deviations  for  the 
smallest  target.  Likewise,  the  maxima  were  taken  to  be  the  mean  plus  two  standard  deviations 
for  the  largest  relevant  target.  The  standard  deviation  values  are  also  required  in  our  library 
target  matching  algorithm. 

Table  6. 1 :  Statistics  and  min/max  values  for  each  of  the  3  blind  areas  at  the  APG  test  site. 


SNR'  Fit'  Ma  Scatter  Plot  '  APG  Cal  Grid 


Figure  6.18:  Scatter  plot  illustrating  the 
relationship  between  static  data  SNR  and  Fit. 
The  3rd  dimension  is  a  figure-of-merit  metric 
Ma  that  can  be  used  as  a  penalty  function  for 
discrimination  of  data  sets  with  low  SNR 
and/or  low  Fit  statistic.  The  data  are  from  the 
cal  lanes  at  APG.  Color  has  been  modulated 
byMa. 
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Library  Curve  Matching 

For  completeness,  we  have  included  polarizability  plots  for  each  of  the  6  ordnance  types  as 
Figure  6.19.  In  these  plots,  curves  representing  5  or  6  different  static  data  sets  from  the 
Calibration  Grid  are  plotted.  The  targets  were  seeded  at  2  depths  and  2  or  3  different  attitudes. 
Also  included  in  the  plots  are  the  geometric  average  polarizability  curves  representing  Px(t)  and 
PT(t)  (dashed  turquoise  and  magenta  curves,  respectively).  We  used  the  6  pairs  of  average 
“type”  curves  as  library  curves  in  a  curve-matching  algorithm  for  discrimination.  Table  6.2  lists 
the  3  areas  of  interest  at  APG  and  the  ordnance  targets  they  contain. 


Table  6.2:  The  6  targets  of  interest  (TOI)  at  the  APG  demonstration  site.  The  “X”  designates 
that  targets  of  the  indicated  type  have  been  seeded  in  a  particular  area. _ 


Target  Type 

Blind  Grid 
(BG) 

Indirect  Fire  Area 
(IF) 

Direct  Fire  Area 
(DF) 

25mm  M792 

X 

X 

37mm  M63M1 

X 

X 

105mm  HEAT  M486 

X 

X 

60mm  Mortar  M49A3 

X 

X 

81mm  Mortar  M324 

X 

X 

105mm  M60 

X 

X 

Matching  with  library  curves  is  performed  in  log  space.  Rather  than  match  the  3  principal 
polarizability  curves  individually,  we  characterize  the  curve  set  with  2  transient  curves,  dPx/dt 
and  the  gate  by  gate  ratio  dPx/dt/V(  dPy/dt  dPz/dt),  and  a  scalar  measure  of  the  single-axis 
symmetry  characteristic  Poe ■  A  flow  diagram  of  the  process  is  illustrated  in  Figure  6.20.  The 
process  has  1  global  parameter  (n)  plus  1  type-specific  parameter  (cr;).  We  match  the  two 
transients  derived  from  the  3  principal  polarizability  transients18  with  similar  transients  derived 
from  the  Target  “Type”  Library.  The  matching  function  for  dPx/dt  depends  on  amplitude  as 
well  as  shape  and  thus  delivers  an  estimate  of  the  ratio  of  the  amplitude  of  the  two  major 
polarizability  curves.  The  second  transient  (labeled  Pr  in  Figure  6.20)  is  a  ratio  of  two 
polarizability  rates  and  thus  is  independent  of  amplitude.  Therefore,  the  matching  simply 
provides  an  estimate  of  shape.  The  two  shape  matching  results  are,  respectively,  tpx  and  (pr. 
These  outputs  take  a  value  of  1  when  the  shape  is  perfect.  We  use  the  amplitude  ratio  parameter 
(g)  in  a  Gaussian  penalty  function  to  effectively  diminish  the  value  of  (px  when  the  amplitude 
ratio  of  the  library  type  and  the  unknown  type  is  different  than  1.  If  there  is  a  perfect  match,  g= 
<px=  (pr=l ■  The  behavior  of  the  gain  penalty  function  is  controlled  by  two  parameters  (»,  and  a). 
We  choose  the  value  n  such  that  there  is  a  substantial  penalty  on  the  output  value  (@,j  when  the 
principal  polarizability  curve  set  is  outside  the  expected  range  of  amplitudes.  We  chose  the  value 
of  n=2  which  penalizes  a  good  shape  fit  by  a  factor  of  0.6  when  the  gain  is  2  geometric  standard 
deviations  away  from  the  geometric  mean.  The  value  of  a  is  related  to  the  standard  deviations 


18  The  ratio  transient  Pr(t)  is  the  ratio  of  the  2  dashed  curves  shown  in  Figure  6.17. 
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reported  in  Table  6.1.  The  algorithm  penalizes  a  match  when  the  unknown  target  polarizability 
curves  exhibit  poor  symmetry  characteristics  as  quantified  by  the  scalar  parameter  Poe ■  Poe  is 
always  in  the  range  0<  Poe'S!  and  has  the  value  0  when  the  two  minor  principal  polarizability 
curves  are  identical.  So  the  penalty  function  p=l-  Poe  penalizes  a  good  match  when  the 
unknown  data  set  has  poor  symmetry.19 

The  methodology  outlined  above  produced  good  discrimination  scores  [16].  The  scores  will  be 
discussed  in  more  detail  in  section  7. 


19  Poe  for  ordnance  is  typically  less  than  0. 1 .  Many  obvious  clutter  objects  have  Poe  values  on  the 
order  of  0.2,  so  the  penalty  function  (i-  Poe)"1  is  perhaps  overly  lenient  .  Perhaps  the  exponent 
should  be  2?  We  have  not  tested  this. 
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Figure  6.19:  Polarizability  plots  of  the  6  ordnance  types  of  interest  in  the  recently 
reconfigured  UXO  test  area  at  APG.  All  6  are  present  in  the  Blind  Grid.  The  3  types 
identified  with  an  appended  “(DF)”  have  been  seeded  in  the  “Direct  Fire”  area  and  likewise 
type  names  with  an  appended  “(IF)”  have  been  seeded  in  the  “Indirect  Fire”  area.  The  heavy 
dashed  curves  represent  the  (geometric)  average  of  a  population  of  training  polarization 
curves  extracted  from  MetalMapper  data  acquired  over  the  Calibration  Grid  at  APG. 
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Figure  6.20:  A  simplified  computational  flow  diagram  that  shows  the  process  of  using  library 
polarizability  curves  to  identify  the  target  “type”  of  an  unknown  target.  The  index  i  indicates 


the  ith  library  type.  The  unknown  polarizability  curves  are  matched  with  each  of  the  (N) 


“types”  in  the  Target  “Type”  Library. 


6.4.3.  Discrimination  at  Camp  San  Luis  Obispo 


The  discrimination  at  APG  was  decidedly  more  “principled”  than  the  approach  we  described  for 
YPG.  The  library  matching  does  a  good  job  of  identifying  targets  of  interest  when  the  Fit  is 
good.  But,  as  the  matching  threshold  is  lowered  so  as  not  to  miss  targets  that  are  obviously 
ordnance  (based  on  visual  correlation  with  polarizability  curves),  many  false  positive  decisions 
appear.  For  the  San  Luis  Obispo  demonstration,  we  therefore  added  an  automated  neural 
network  (ANN)  to  our  discrimination  algorithm.  The  discrimination  flow  diagram  for  SLO 
(Figure  6.21)  is  similar  to  the  one  for  APG  (Figure  6.17).  However,  we  have  now  expanded  our 
processing  flow  to  include  an  ANN  that  operates  on  a  set  of  9  scalar  features  derived  from  the 
polarizability  curves.  The  ANN  assigns  a  binary  decision  (i.e.  “TOT’  or  “Clutter”)  to  each  data 
set  not  otherwise  classified  by  rule-based  decision.  As  a  final  step,  we  use  the  library  matching 
algorithm  as  a  means  to  identify  TOl  that  elicit  false  negative  decisions  from  the  ANN. 


In  the  following  paragraphs,  we  expand  on  our  use  SNR  and  Fit  parameters  in  deciding  whether 
data  are  analyzable,  and  we  review  our  rule -based  decisions.  Then  we  provide  a  thorough 
explanation  of  the  ANN  that  we  used  and  results  from  that  training.  With  regard  to  library  curve 
matching,  there  have  been  no  substantive  changes  to  the  algorithm  that  we  outlined  in  section 
6.4.2.  We  used  5  library  “type”  curves  in  our  matching.  We  present  those  library  types  and 
discuss  their  significance. 
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Figure  6.21:  Processing  diagram  for  discrimination  with  MetalMapper  static  data  sets  from 
SLO. 


Data  Quality  and  Rule-Based  Decisions 

The  overriding  objective  for  the  rule-based  decisions  is  to  select  static  data  sets  that  we  believe 
can  be  analyzed.  Assuming  we  have  an  analyzable  data  set,  MMRMP  provides  reliable 
estimates  of  target  position  and  target  size  (as  expressed  by  the  integrated  principal  polarizability 
values,  Pox,  Poy,  Poz)-  The  size  estimates  can  be  used  to  eliminate  those  targets  that  are  either  too 
large  or  too  small  to  be  a  target  of  interest  (TOI).  Our  experience  in  the  analysis  of  AOL2  and 
MM  static  data  at  YPG,  and  at  APG  suggest  that  we  obtain  reliable  estimates  of  position  and 
target  size  when  the  SNR  >  10  and  the  Fit  >  85%  and  our  first  inclination  was  to  screen  the  data 
and  designate  all  data  sets  for  targets  whose  SNR  and/or  Fit  falls  below  these  thresholds. 
However,  after  examining  polarizability  curves  and  scalar  meta-parameters  (see  section  6.3.2) 
for  many  data  sets  with  Fit  statistic  values  less  than  85%,  we  are  convinced  that  Fit  =  85%  more 
likely  defines  the  upper  end  of  a  transition  zone.  In  this  zone,  the  reliability  of  the  data  certainly 
declines  in  an  undefined  but  monotonic  way.  We  believe  that  we  can  still  rely  on  MMRMP  to 
produce  rough  estimates  of  target  position  and  size.  A  look  at  the  distribution  of  the  SNR  and 
the  Fit  for  the  2492  static  data  sets  that  we  acquired  is  instructive  (Figure  6.22).  The  SNR 
histogram  (Figure  6.22-Top)  that  there  are  only  15  data  sets  with  an  SNR  of  less  than  25dB. 
Thus  at  SLO,  SNR  by  itself  will  be  ineffective  as  a  means  of  identifying  “Cannot  Analyze”  data 
sets.  Secondly,  the  distribution  of  Fit  statistics  indicates  that  a  total  of  771  data  files  have  Fit 
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statistics  falling  below  the  85%  threshold.  In  an  effort 
to  minimize  so  far  as  possible  the  number  of  data  points 
we  place  immediately  into  the  “Cannot  Analyze” 
category,  we  adopted  a  2-stage  approach  to  screening 
our  data  for  analyzability  at  SLO.  In  the  first  stage,  we 
simply  screen  for  Fit  <  75%.  Based  on  the  distribution 
of  Fit  (Figure  6.22  bottom),  this  threshold  classified 
approximately  4%  of  all  targets  as  “Cannot  Analyze”. 

The  threshold  (75%)  is  purposefully  set  to  a  level  that  is 
perhaps  lower  than  our  confidence  in  the  data. 

However,  in  a  second  stage,  we  apply  a  figure  of  merit 
function  that  “penalizes”  the  output  of  our  ANN  for  low 
levels  of  the  Fit  parameter.  Likewise,  we  apply  the 
same  Figure  of  Merit  (FOM)  function  to  output  metric 
from  our  library  matching  algorithm. 

Figure  of  Merit  Penalty  Function  (MJ 

In  section  6.4.2,  we  introduced  the  FOM  without 
explicitly  defining  it.  We  assume  that  the  target  is 
isolated  from  adjacent  targets  so  that  the  secondary 
fields  we  measure  are  primarily  due  to  the  target  of 
interest.  Generally  speaking,  when  the  SNR  is  sufficiently  high,  the  MetalMapper  data  acquired 
over  an  isolated  target  will  always  provide  useful  parameters  and  the  Fit  statistic  will  be  greater 
than  85%.  A  data  set  with  a  high  SNR  and  a  low  Fit  statistic  (i.e.  «85%)  suggests  that  the  field 
is  not  analyzable.  Our  figure-of-merit  function  (Ma)  combines  SNR  and  Fit  into  a  parametric 
function  defined  by  three  parameters  (snro-,a,b).  The  mathematical  formula  we  use  is 

Ma(snr,fit ;  snr0,  a,  b )  =  /(snr;  snr0)g(fit ;  a,  b ) 

Where, 

/(snr;  snr0)  =  Max  [o.l  +  Min  [o,  log10  (  m  (6.3) 


Distribution  of  SNR  for  2492  MM  Data  Sets 
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Figure  6.22:  Histograms  showing 
the  distribution  of  SNR  and  Fit 
statistic  for  2492  MM  static  files 
acquired  at  SLO. 
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We  used  data  sets  and  ground-truth  from  the  calibration 

lanes  at  both  YPG  and  APG  to  determine  that  Table  6.3:  Parameter  values  for 
analyzability  of  target  parameters  from  data  sets  with  an 
SNR  less  than  10  and/or  a  Fit  statistic  of  less  than  90% 
fall  off  rapidly.  The  values  for  the  3 -parameter 
analyzability  function  (equation  6.3)  have  been  chosen 
by  analyzing  MM  data  sets  from  the  calibration  lanes  at 
the  Standardized  UXO  Technology  Demonstration 
Site(s)  located  at  YPG  and  APG.  These  values  are 
provided  below.  The  behavior  of  the  2  functions  for  the  parameter  values  we  used  at  SLO  shown 
in  Figure  6.23:  Plots  showing  the  figure  of  merit  functions  defined  in  equation  (3-1)  for  the 
parameter  values  in  Table  3-1.  As  we  show  in  Figure  6.21,  the  penalty  function  is  applied  as  a 


figure  of  merit  function  Mg 


Parameter 

Value 

snro 

20 

a 

2.5 

b 

8 

multiplier  to  the  outputs  of  the  ANN.  While  not  changing  the  classification,  the  function 
decreases  the  confidence  level  for  each  of  the  classes.  Thus,  all  targets  with  a  fit  value 
substantially  lower  than  85%  will  have  low  confidence  and  can  be  screened  using  two  thresholds 
(Tel,  and  To).  Targets  falling  within  these  boundaries  are  designated  as  “ Ambiguous ”  (i.e., 
Analyzable  -  Can’t  Decide). 


We  have  calculated  the  cumulative  number  of  data  points  as  a  function  of  Fit  using  all  2492 
static  points  that  we  measured  at  SLO  and  plotted  the  results  together  with  the  Fit  histogram 
(Figure  3.1)  in  Figure  3.3.  If  we  use  a  cut-off  value  of  75%  below  which  all  points  are  declared 
as  “Cannot  Analyze-Must  Dig”,  111  data  points  (-4.5%)  will  be  excluded. 
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Figure  6.24:  Fit  histogram  (left)  together  with  the  cumulative  static  data  points  as  a  function 
of  the  MMRMP  Fit  statistic. 


Target  Size 


As  a  result  of  initial  reconnaissance  at  the  SLO 
demonstration  site,  the  ESTCP  program  office 
identified  4  targets  of  primary  interest.  Those 
targets  were  the  2.36-in  “Bazooka”  rocket,  60mm 
mortar,  81mm  mortar,  and  4.2  in  mortar.  Our  static 
measurements  over  specimens  of  these  4  targets  in 
the  test  pit  and  in  the  test  lanes  at  SLO  have 
provided  us  with  14  test  data  sets  for  each  of  these 
targets  at  different  depths  and  attitudes.  From  the 
target  parameters  we  have  extracted  from  these 
data  sets,  we  estimated  a  threshold  target  size  such 
that  any  target  that  has  been  placed  in  the  Can 
Analyze  category  whose  electromagnetic  size  (i.e., 
P0)  falls  below  P0m;n  falls  into  the  category  of 
high-confidence  not  munition  (i.e.,  “clutter”).  From 
the  training  data  acquired  in  the  test  pit,  the  test 
strip,  and  the  training  dig,  we  have  determined  the 


Table  6.4:  Table  summarizing  the 
statistics  of  the  4  MOI’s  of  interest  at 
SLO.  We  use  this  table  to  define  the 
range  of  target  sizes  that  may  be 
munitions. 


Target 

POx  (Test  Pit  &  Strip) 

POx  (Training  Dig) 

GM 

GSD  |  Hi 

Lo 

GM 

GSD 

1  Hl 

Lo 

2.36-in  Rocket 

5159 

0.133  9419 

2825 

3617 

0.326 

15751 

831 

4.2-in  Mortar 

19546 

0.076  27531 

13877 

19505 

0.084 

28496 

13351 

81mm  Mortar 

6067 

0.107  9817 

3751 

7168 

0.029 

8170 

6289 

60mm  Mortar 

2585 

0.050  3241 

2062 

1534 

0.114 

2566 

917 

5-in  Rocket 

3612 

N/A 

N/A 

N/A 

3-in  Mortar 

1259 

N/A 

N/A 

N/A 

37mm  Mortar 

615 

N/A 

N/A 

N/A 

Comments: 

GM  Geometric  Mean 
GSD  Geometric  Standard  Deviation 
Hi  Expected  high  value  (95%  confidence) 
Lo  Expected  low  value  (95%  confidence) 
Units  All  values  in  units  of  cm3 


minimum  value  of  POx  based  on  measured  statistics  of  PO-values  for  each  of  the  4  TOI.  Table 


6.4  summarizes  these  statistics.  The  table  also  contains  the  static  POx  values  observed  over  3 


additional  targets,  a  5-in  rocket,  a  3-in  (Stokes)  mortar,  and  a  37mm  projectile  that  were  found 
during  subsequent  digging  (shaded  grey  in  the  table).  There  are  no  statistics  for  these  targets 
because  only  one  instance  of  each  type  has  thus  far  been  identified.  Figure  6.25  is  a  graphic  that 
summarizes  the  distribution  of  polarizability  for  the  static  data  measured  over  the  TOI  included 
in  the  Training  Dig  (6.25A  &  6.25C)  and  over  targets  placed  in  the  test  pit  and  buried  in  the  test 
strip  at  SLO  (6.25B  &  6.25D).  Figures  6.25A  and  6.25C  summarize  results  for  approximately 
200  targets  excavated  at  SLO  for  which  the  ground-truth  was  released  to  demonstrators  for  use 
as  training  data.  For  reference,  we  have  generated  scatter  plots  in  Figures  6.25B  and  6.25D  that 
contain  only  results  measured  in  the  test  pit  and  test  strip.  The  statistics  were  developed  in  log 
space  and,  therefore,  the  corresponding  values  in  linear  space  are  the  “ Geometric  Mean  ”  and  the 
“ Geometric  Standard  Deviation ”.  The  “hi/lo”  values  in  the  table  are  estimates  for  a  95% 
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confidence  interval  assuming  a  lognormal  distribution.  Based  on  the  hi/lo  values  of  the  4  TOI  in 
Table  6.4  for  the  POx  parameter20  we  set  our  decision  rule  for  size  based  on  POx  values  for  all 
TOI  staying  in  the  range  500  <  P0x  <  55,000.  The  37mm  projectile  has  a  Pqx  that  is 

uncomfortably  close  to  the  lower  limit  of  our  target  size  range.  Further  work  at  SLO  will  require 
a  change  to  the  lower  limit  of  the  size  range  (perhaps  Poxmin=400)  and  a  recalculation  of 
detection  thresholds  using  37mm  mortar.  In  establishing  this  range,  we  set  the  minimum  value 


£ 

O 

c u 


SLO  Training  Dig  Data  -  6  Plot 


Longitudinal  P0  (cm3) 


SLO  Test  Pit  &  Strip  -  p  Plot 


SLO  Training  Dig  -  POx  vs  ix 


SLO  Test  Pit  &  Strip  -  POx  vs  ix 


Figure  6.25:  Polarizability  scatter  plots  based  on  ground  truth  from  the 
training  dig  (Top  -  111  targets)  and  training  data  from  the  test  pit  and 
test  strip  at  SLO  (67  targets).  The  vertical  bars  (green)  represent  the 
parameter  P0e-  Targets  with  large  Poe  do  not  exhibit  an  axis  of 
symmetry  that  is  generally  associated  with  ordnance  items 


to  be  substantially  below  the  “lo”  value  for  POx  for  the  60mm  mortar  (and  2.36-in  rocket  pieces 
that  were  classified  as  “TOI”)  and  the  high  value  at  twice  (2  x)  the  “hi”  value  of  the  4.2 -in 
mortar.  Targets  falling  outside  this  target  size  range  were  classified  as  “ High-Confidence  Not 


Munitions ”. 


20  We  chose  Pox  because  it  is  the  largest  and  most  reliable  of  the  3  scalar  principal  polarizability 
(“betas”)  estimates. 
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Target  Types  for  Library  Matching 

Initial  reconnaissance  dig  information,  later  buttressed  by  results  from  the  training  dig,  identified 
4  targets  of  primary  interest.  Consequently,  the  ESTCP  and  the  MetalMapper  demonstration 
plans  provided  for  the  acquisition  of  static  data  over  specimen  targets  of  those  types.  And,  as  we 
mentioned  in  the  previous  section,  it  was  not  until  after  the  training  dig  was  completed  and  those 
results  had  been  distributed  that  digging  unearthed  other  munitions  types  (e.g.,  3-in  Stokes 
mortar,  37mm  projectile,  and  5-in  Rocket).  We  developed  our  initial  “type”  polarizability  curve 
library  from  the  static  measurements  we  acquired  with  the  MetalMapper  in  the  test  pit  and  the 
test  strip.  Using,  the  specimen  targets  provided  at  SLO  we  measured  the  free-air  response  of 
each  target  placed  at  4  different  attitudes  and  2  different  depths.  In  addition,  we  took  static 
measurements  over  each  of  the  targets  buried  in  the  test  strip  several  times  during  the  course  of 
our  demonstration.  In  total,  we  were  able  to  collect  14  independent  data  sets  for  each  of  the  4 
static  target  types.  The  principal  polarizability  curves  for  each  of  the  4  type  ensembles  are 
shown  in  Figure  6.26.  As  a  result  of  the  training  dig,  we  learned  that  the  60mm  mortars  that  we 
unearthed  often  had  neither  the  fuses  nor  the  fins  that  were  present  on  the  specimen  that  we  used 
for  test  pit  measurements  and  on  those  that  were  buried  in  the  test  strip.  The  differences  in  the 
principal  polarizability  curves  for  the  two  types  of  60mm  targets  can  be  seen  in  the  average 
principal  polarizability  curves  shown  Figure  6.27.  The  60mm  types  with  no  fins  and  fuses 


Figure  6.26:  Principal  polarizability  curves  for  the  4  munitions  of  primary  interest  at  SLO. 
Each  curve  set  is  derived  from  an  ensemble  of  14  static  measurements  over  targets  at  different 
depths  and  orientations.  The  heavy  curves  (red/green/blue)  are  the  geometric  mean  curves 
computed  by  taking  the  geometric  mean  of  their  respective  ensemble. 
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exhibit  a  smaller  aspect  ratio  Pr  at  later  time.  In  the  scalar  features,  this  manifests  itself 
primarily  in  smaller  values  for  the  two  moments  (Pox,  and  Pix)  and  in  the  aspect  ratio  parameter 
Por.  Mortars  with  fins  often  exhibit  a  size  as  indicated  by  the  feature  Pox  that  is  twice  that  of  a 
mortar  without  fins  and  a  fuse.  During  the  training  dig  at  SLO,  the  excavated  60mm  mortars 
were  mostly  ones  without  fins  and  fuses.  To  insure  that  our  library  matching  algorithm 
recognized  both  types  of  60mm  mortars,  we  included  both  type  curves  for  both  configurations  of 
the  60mm  mortars  in  our  library. 
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Figure  6.27:  Principal  polarizabity  curves  for  2  mortar  types: 
Mortars  with  fins  (and  fuses?)  exhibit  characteristically  larger 
values  of  Pqx  and  a  larger  aspect  value  (Por). 
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Artificial  Neural  Net  (ANN)  Classifier 

The  introduction  of  an  ANN  to  generate  a  binary  decision  regarding  whether  a  particular  target  is 
a  target  of  interest  represents  the  most  significant  change  our  classification  procedure.  For  this 
purpose,  we  used  the  commercially  available  software  package  Statistica  version  6. 0/9.0.  We 
analyzed  scalar  polarizability  parameters  extracted  from  static  MM  data  sets  collected  over  2,177 
targets.  The  23  polarizability  parameters  we  examined  are  listed  in  Table  6.5.21 


Table  6.5:  Table  of  scalar  target  parameters  extracted  from  static  data  sets  acquired  with  the 
MetalMapper  _ _ _ _ _ _ 


POx 

POy 

POz 

NormPO 

POT 

POR 

POE 

Plx 

Ply 

Plz 

NormPl 

PIT 

P1R 

PIE 

Klx 

Gamma  lx 

Kly 

Gamma  ly 

Klz 

Gamma  lz 

Plx/POx 

Ply/POy 

Plz/POz 

Our  objective  was  to  combine  a  basic  statistical  analysis  of  the  data  with  expert  knowledge  about 
the  data  and  the  ANN  choice  of  parameters  to  select  a  subset  of  parameters  to  use  for  the  ANN 
analysis.  Input  parameters  should  be  selected  to  provide  the  best  separation  between  clutter  and 
TOI  signatures.  In  order  to  keep  the  number  of  connection  weights  in  the  ANN  small,  we  want 
to  identify  the  smallest  set  of  input  parameters  that  can  provide  effective  classification. 

We  visually  classified  2,177  targets  as  UXO  or  clutter.  This  was  a  rough  classification  so  the 
statistics  for  the  two  categories  no  doubt  included  some  incorrectly  classified  targets.  The 
statistics  for  TOI  are  shown  in  Table  6.6  and  for  clutter  in  Table  6.7.  There  is  good  separation  of 
the  mean  for  each  parameter  in  the  two  classes  and  there  is  little  overlap  of  minimum  and 
maximum  values  of  each  parameter  between  classes.  Next  we  look  at  the  correlation  of 
parameters.  Ideally  we  want  parameters  that  are  not  correlated  with  each  other  as  input 
parameters  as  they  are  redundant. 


21  See  6.3  for  definitions.  The  P0  parameters  represent  the  0th  moment  of  the  respective 
component  of  the  principal  polarizability  transients.  Likewise,  the  PI  parameters  represent  the 
first  moment  in  of  the  principal  polarizability  transient.  Kl,  and  Gammal  are  parameters  arising 
from  the  parametric  fit  of  equation  6.1  to  a  polarizability  transient. 
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_ Table  6.6:  TOI  Statistics _ 

Descriptive  Statistics  (TEM_Edited_AIISections_TestwOutp 


Variable 

Valid  N 

Mean 

Minimum 

Maximum 

Std.Dev. 

POX 

315 

9620 

43.6 

907142 

51360 

POY 

315 

3315 

25.3 

356674 

20182 

POZ 

315 

2855 

21.8 

26377C 

14992 

NormPO 

315 

10612 

61.6 

1009801 

57175 

P0 T 

315 

3085 

24.9 

310222 

17584 

P0 R 

315 

4 

1.2 

19 

3 

P0 E 

315 

0 

0.0 

0 

0 

P1X 

315 

1279698C 

13806.9 

732048526 

43005282 

PI  Y 

315 

2674054 

-1426.4 

238871126 

13638102 

P1Z 

315 

2325361 

-10441.2 

193343616 

11089876 

NormPI 

315 

1333173£ 

16603.9 

793937162 

46443712 

P1 T 

315 

2499708 

2161.9 

216107372 

12362255 

P1 R 

315 

8 

1.2 

178 

12 

P1 E 

315 

0 

-0.4 

0 

0 

Klx 

315 

17009 

0.0 

100000C 

57441 

gammalX 

315 

161246 

0.0 

1000000C 

1251515 

Kly 

315 

13610 

131.1 

100000C 

57044 

gammal  Y 

315 

2101627 

634.0 

1000000C 

4073332 

Klz 

315 

12827 

97.5 

86281 C 

49486 

gammalZ 

315 

23524 1C 

0.9 

1000000C 

4244462 

PIx/POx 

315 

1237 

271.7 

2323 

441 

Ply/POy 

315 

723 

-30.7 

1144 

194 

Plz/POz 

315 

681 

-33.5 

1494 

221 
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_ Table  6.7:  Clutter  statistics _ 

Descriptive  Statistics  (TEM_Edited_AIISections_Test  wOutp 


Variable 

Valid  N 

Mean 

Minimum 

Maximum 

Std.Dev. 

POX 

1862 

1429 

13  1.430998E+05 

33733 

POY 

1862 

819 

1 

8.271481  E+05 

19635 

POZ 

1862 

472 

0 

4.301 327E+05 

10272 

NormPO 

1862 

1730 

16 

1.707906E+05 

40360 

P0 T 

1862 

646 

1 

6.286404E+05 

14952 

P0 R 

1862 

4 

1 

5.01 3927E+01 

4 

P0 E 

1862 

0 

0 

8.71 8334E-01 

0 

P1X 

1862 

1205184 

-59510 

1.157193E+05 

27649922 

PI  Y 

1862 

544935 

-433417 

5.325576E+05 

12884785 

P1Z 

1862 

30748C 

-1355761 

3.008459E+05 

7211944 

NormPl 

1862 

1375617 

3599 

1. 308901 E+OS 

31 34485C 

P1 T 

1862 

426207 

-545712 

4.16701 8E+05 

10045975 

P1 R 

1862 

5 

-340 

4.233777E+02 

20 

P1 E 

1862 

0 

-35 

1.943966E+01 

1 

Klx 

1862 

4497 

15 

1.000000E+05 

31990 

gammalX 

1862 

1545149 

175 

1.000000E+07 

3610195 

Kly 

1862 

3706 

6 

1.000000E+05 

30333 

gammal  Y 

1862 

3756215 

260 

1.000000E+07 

4836931 

Klz 

1862 

3208 

0 

1.000000E+05 

28577 

gammalZ 

1862 

4034865 

0 

1.000000E+07 

4900365 

Plx/POx 

1862 

572 

-372 

2.743929E+03 

320 

Ply/POy 

1862 

473 

-13619 

1.946209E+05 

511 

Plz/POz 

1862 

692 

-21772  1.830876E+05 

5651 

For  clutter  targets,  POX  is  highly  correlated  with  POY,  POZ,  NormPO,  PO  T,  P1X,  P1Y,  P1Z, 
NormPl,  Klx,  Kly,  and  Klz.  The  parameters  P0_R,  PO  E,  P1_R,  PI  E,  ylx,  yly,  ylz,  Plx/POx, 
Ply/POy,  and  Plz/POz  are  not  correlated  with  other  parameters. 


Table  6.8:  Correlation  matrix  for  the  9  scalar  parameters  (features) 
selected  for  use  in  discrimination  at  SLO. 


Gtr  refet  bns  ( TB/ljat  edJ'BBct  bns_Test  _vO  ut  put ) 
Marked  oorrefetbns  are  sgrfcart  at  p  <  .05000 
IT2177  ( Chsevce  debt  fcn  of  msshg  dat  a) 


VrEbte 

rax 

POY 

RE 

ro R 

P0 E 

P1 E 

Plx/Rk 

Ply/FCy 

Plz/Rk 

PCX 

1.00 

0.99 

1.00 

-0.01 

-0.01 

-0.00 

0.07 

0.03 

0.00 

PCY 

0.99 

1.00 

0.99 

-0.02 

0.01 

0.00 

0.05 

0.03 

0.00 

RE 

1.00 

0.99 

1.00 

-Q  02 

-Q  00 

-0.00 

0.08 

0.03 

0.00 

F0 R 

-0.01 

-Q  02 

-Q  02 

1.00 

-Q  32 

-0.08 

0.07 

-0. 12 

0. 12 

P0 E 

-0.01 

0.01 

-0.00 

-0.32 

1.00 

0.15 

-0.34 

-0.01 

0.00 

PIE 

-0.00 

0.00 

-0.00 

-0.08 

0.15 

1.00 

-0.02 

0.03 

-0.08 

Plx/Rk 

0.07 

0.05 

0.08 

0.07 

-Q  34 

-0.02 

1.00 

0.32 

0.00 

Ply/FCy 

0.03 

0.03 

0.03 

-Q  12 

-Q  01 

0.03 

0.32 

1.00 

0.01 

Plz/Rk 

0.00 

0.00 

0.00 

0. 12 

0.00 

-0.06 

0.00 

0.01 

1.00 
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The  selection  of  parameters  for  the  ANN  is  dependent  on  the  total  number  of  parameters  and  the 
quality  of  the  training  data  and  is  not  unique.  We  ran  a  hierarchical  classification  tree, 
discriminant  analysis,  and  4  ANNs.  For  each  ANN,  we  looked  at  the  sensitivity  of  the 
classification  to  the  input  parameters.  The  list  below  (Table  6.9)  indicates  in  how  many  of  the 
trials  each  parameter  was  selected  as  important  (total  of  6  trials): 


Table  6.9:  Results  of  ANN 


(and  other)  parameter  selection 


Param 

Sel  Cnt 

Param 

Sel  Cnt 

Param 

Sel  Cnt 

Param 

Sel  Cnt 

POx 

2 

Plx 

0 

Klx 

3 

Gammalx 

1 

POy 

0 

Ply 

0 

Kly 

2 

Gammaly 

2 

POz 

0 

Plz 

0 

Klz 

3 

Gammalz 

2 

NormPO 

0 

NormPl 

1 

Plx/POx 

5 

POT 

0 

PIT 

0 

Ply/POy 

4 

POR 

3 

P1R 

2 

Plz/POz 

3 

POE 

6 

PIE 

4 

Using  parameters  that  were  selected  at  least  twice  in  our  analyses,  the  input  parameter  set 
(feature  vector)  would  be: 

(P0x,P0R,  POE,  P1R,  PIE,  Klx,  Kly,  yly,  Klz,  ylz,  Plx/POx,  Ply/POy,  Plz/POz) 


The  various  K1  and  yl  parameters  are  derived  from  the  fit  of  their  respective  principal 
polarizability  curves  with  a  parametric  relation  (equation  6.1).  The  K  and  y  parameters  have  the 
same  units  and,  indeed,  the  same  physical  meaning  (i.e.,  target  size,  and  time  decay)  as  their 
more  robust  counterparts  (PO,  and  P1/P0),  which  are  calculated  by  simple  numerical  integration 
and  ratios.  Therefore,  we  eliminated  K’s  and  y’s  from  our  list  of  features  where  they  provide 
redundant  information  (e.g.  Klx~P0x;  ylz~Plz/POz,etc)  and  have  substituted  POy  for  Kly,  and 
POz  for  Klz.  Our  final  feature  vector  includes  9  robust  parameters  that  capture  (in  scalar  form) 
the  properties  of  target  size  (PO  -  3  axes),  target  aspect  (POR),  target  symmetry  (PO  E),  and 
target  polarizability  time  persistence  (P1/P0  -  3  axes): 


(POx,  POy,  POz,  POR,  POE,  PIE,  Plx/POx,  Ply/POy,  Plz/POz) 

In  our  initial  experimentation,  the  only  training  data  we  had  were  those  that  we  acquired  in  the 
test  pit  and  over  the  test  strip.  Therefore,  we  selected  targets  to  use  as  training  data  by  visual 
inspection  of  all  the  target  data  with  the  goal  of  selecting  clear  examples  of  the  4  types  of  UXO 
together  with  representative  clutter.  The  resulting  training  set  contained  more  examples  of 
clutter  than  UXO  because  there  is  more  variability  in  the  clutter.  We  chose  257  examples  of 
clutter  and  165  examples  of  UXO.  There  were  at  least  76  targets  that  we  classed  as  clutter  based 
on  POx  amplitude  (<500  cm3)  but  which  had  good  symmetry.  Based  on  our  visual  classification 
we  found  315  TOI  and  1,862  clutter.  Thus  we  used  approximately  52%  of  the  UXO  examples 
and  14%  of  the  clutter  in  our  training  set. 
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After  the  results  of  the  training  dig  were  distributed  to  those  involved  in  the  classification 
processing,  we  had  (barely)  sufficient  data  (when  added  to  the  test  pit  and  test  strip)  so  that  we 
were  able  to  train  a  network  totally  on  data  for  which  the  ground  truth  was  known.22 

ANN  -  We  used  a  feed- forward  multi-layer  perceptron  architecture  with  backpropagation  and 
quasi-Newton  training  algorithms.  We  ran  backpropagation  for  55  epochs  and  then  switched  to 
quasi-Newton  for  20  epochs.  Backpropagation  is  a  fast  algorithm  but  often  has  trouble 
converging  to  the  minimum.  Quasi-Newton  is  computationally  more  intensive  and  needs  to  run 
when  we  are  closer  to  the  minimum.  Hence,  we  use  backpropagation  to  get  close  to  the 
minimum  error  and  then  quasi-Newton  for  the  final  few  epochs.  Input  parameters  were  linearly 
scaled  to  a  range  of  [-1,1]  by  taking  the  minimum  and  maximum  value  for  each  parameter  and 
scaling  that  range  to  the  ANN  input  range.  We  used  a  starting  step  size  of  0.02  and  momentum 
of  0.4.  The  step  size  and  momentum  were  automatically  adjusted  each  epoch  with  a  final  step 
size  of  0.1,  and  a  final  momentum  of  0.3.  We  had  9  input  parameters,  1  hidden  layer  with  6 
hidden  nodes,  and  1  output.  We  used  a  hyperbolic  tangent  activation  function  for  the  hidden 
layer  and  a  logistic  activation  function  for  the  output.  We  used  an  entropy  calculation  to 
determine  the  output  error  during  training.  This  error  is  the  sum  of  the  products  of  the  target 
value  and  the  logarithm  of  the  error  value  on  each  output  unit. 

The  output  was  coded  as  0  for  clutter  and  1  for  TOI.  We  set  a  threshold  of  0.5  and  all  outputs 
less  than  0.5  were  classed  as  clutter  and  those  greater  than  0.5  were  UXO.  The  closer  an  output 
is  to  0  the  more  confidence  we  have  in  the  clutter  classification  and  the  closer  the  output  is  to  1 , 
the  more  confidence  we  have  in  the  UXO  classification. 

Statistica  randomly  selects  half  of  targets  for  training,  a  quarter  for  “selection”,  and  a  quarter  for 
testing.  The  samples  are  chosen  to  have  similar  means,  ranges,  and  standard  deviations.  The 
training  samples  are  used  to  adjust  the  connection  weights.  Periodically  the  weights  are  frozen 
and  the  selection  targets  are  passed  through  the  network  once  and  their  outputs  are  compared 
with  their  desired  output.  This  is  done  to  monitor  for  overtraining.  The  training  process  can 
proceed  until  all  training  samples  are  learned  perfectly  but  this  runs  the  risk  of  learning  noise  or 
irrelevant  information  in  the  data  and  the  resulting  network  will  not  generalize  or  test  well.  We 
monitor  the  training  and  selection  error  to  make  sure  that  they  track  each  other.  If  the  training 
error  is  improving  and  the  selection  error  degrades,  we  have  over  trained  and  must  return  to  the 
best  previous  network  configuration.  Once  we  have  the  “best”  network,  the  weights  are  frozen 
and  the  test  data  are  presented  to  the  network.  We  monitor  the  training,  selection,  and  test  errors. 

Classification  Results  -  Training  &  Test 

As  we  indicated  above,  we  ended  up  training  two  networks: 

1.  ANN  Network  1:  This  ANN  was  trained  with  test  pit  and  test  strip  data  (58  data  points), 
data  from  the  test  dig  (208  points),  and  12  data  points  that  were  visually  classified  as 
60mmNF  (No  Fins).  Although  we  had  MM  static  points  associated  with  only  154  of  the 
targets  in  the  training  dig,  many  of  those  targets  had  2  or  more  MM  data  points  associated 


22  The  program  office  allowed  us  to  submit  a  target  list  resulting  from  the  two  ANN’s  (i.e.,  one 
trained  with  known  data  only,  and  one  trained  with  a  training  set  “augmented”  with  visually 
classified  data  points. 
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with  the  target.  By  using  these  (54)  repeated  data  points,  we  “augmented”  our  training 
set  so  that  it  had  278  targets. 

2.  ANN  Network  2:  This  ANN  was  trained  with  a  total  of  563  targets  comprised  of  the  58 
points  from  the  test  pit  and  test  strip,  the  208  points  (as  above)  from  the  training  dig,  and 
296  points  that  were  visually  classified. 


P(p 

A)  Training  ROC  -  ANN  Network  1  B)  Training  ROC  -  ANN  Network  2 


Figure  6.28:  ROC  curves  showing  the  results  of  applying  ANN  and  the  ANN  +  Library 
Curve  matching  to  their  respective  training  sets  after  training.  The  type  curve  library 
contains  5  types  (60mm,  60mmNF,  2.36-in,  81mm,  4.2-in). 


Figure  6.28  shows  the  ROC  curves  that  result  from  applying  the  two  ANN’s  to  their  respective 
training  data.  Note  that  in  both  cases,  the  ANN  leaves  a  few  “hard”  false  negatives  that  would 
require  us  to  dig  a  high  percentage  of  all  the  targets  before  digging  the  last  TOI.  However,  after 
applying  the  library  matching,  the  hard  false  negatives  are  moved  closer  to  the  knee  (operating 
point)  of  the  curve  thereby  dramatically  reducing  the  number  of  targets  we  need  to  dig  before  we 
dig  the  last  TOI  (100%  detection  point). 

It  is  interesting  to  look  at  the  decision  surfaces  generated  by  the  ANN  only  and  then  how  it’s 
modified  by  applying  library  matching.  In  Figure  6.30,  we  overlay  a  contour  plot  of  the  decision 
metric  on  top  of  a  categorized  beta  plot.  The  categories  here  are  “TOI”  (blue  squares)  and 
“Clutter”  (magenta  diamonds).  In  these  plots,  we  have  emphasized  our  decision  threshold  with  a 
dashed  yellow  line.  Lighter  shaded  areas  reflect  areas  where  Pd  >  0.5  and  hence  the  points  are 
more  probably  “TOI”.  The  darker  areas  indicate  Pd<0.5  and  hence  points  in  these  areas  are  more 
likely  to  be  “Clutter”.  In  the  ANN  Only  plot  (Figure  6.30  left  -  center),  notice  that  there  are 
many  TOI  falling  on  or  near  the  decision  threshold  indicating  that  many  of  the  decisions  are 
either  ambiguous  or  even  worse  are  declared  clutter  (i.e.,  false  negatives).  Most  of  those  points 
represent  targets  with  the  type  “60mmNF”  (60mm  mortars  with  no  fins  and  fuses).  After 
applying  library  matching,  the  decision  boundary  is  moved  so  that  the  targets  in  question  are 
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more  firmly  on  the  “TOI”  side  (Figure  6.30  right).  Overall,  the  differences  are  minor.  However, 
the  benefit  of  using  library  matching  is  that  it  reduces  the  number  of  false  negatives. 


Figure  6.29:  ROC  curves  furnished  by  IDA  after  scoring.  Results  of  applying  ANN  and  the 
ANN  +  Library  Curve  matching  to  the  dig  lists  are  shown.  ANN  Network  2  was  used. 


Figure  6.29  shows  the  ROC  curves  furnished  to  us  by  the  IDA  after  scoring  the  two  target  lists 
that  we  submitted.  Note  that  panels  A)  and  B)  of  the  training  ROC  curves  each  have  two  ROC 
curves.  The  curve  labeled  “ANN  Only”  reflects  the  scoring  based  on  the  ANN  output  only.  The 
2nd  curve  (labeled  “ANN  +  Lib  Matching)  reflects  an  adjustment  of  the  discrimination  scores 
based  on  the  output  from  the  library  curve  matching  algorithm.  The  library  curve  matching 
clearly  identifies  most  of  the  false  negatives  and  changes  their  discrimination  score  so  that  in  the 
final  list  they  usually  fall  on  the  high  side  of  the  operating  point  (nominally  0.5). 

Both  of  ANN’s  yield  results  that  very  quickly  reach  a  value  of  Pd=95%.  Network  1  (Figure 
6. 2 8 A)  has  a  few  false  negatives  bunched  together  near  the  operating  point.  However,  there  are 
two  pathological  false  negatives  occurring  at  approximately  350  and  700  digs  (Figure  6.29A)  and 
at  250  and  300  digs,  respectively,  for  Networks  1  and  2.  These  data  points  arise  from  2.36-in 
rocket  parts.  The  difference  between  the  relative  positions  in  each  dig  list  reflects  the  ANN 
scores.  Network  1  indicates  a  higher  probability  that  the  targets  are  clutter  than  does  Network  2. 
The  library  curve  matching  in  both  cases  is  no  help  since  in  both  cases  the  polarizability  curves 
exhibit  clutter-like  properties  (Figure  6.31). 
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Figure  6.30:  Categorized  “beta”  plots  overlaid  with  a  contour  plot  of  the  ANN  only  (left) 
and  ANN  plus  Library  Matching  (right)  decision  surface.  The  decision  threshold  contour  line 
(dashed  yellow) 


SLOStatAO  1718-Principal  Polarizability  Curves  (Type  1) 
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Figure  6.31:  Polarizability  curves  and  dig  photos  for  2  targets  in  the  MM  target  list.  Both 
targets  were  classified  as  “Clutter”  with  relatively  unambiguous  scores. 
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6.5.  DATA  PRODUCTS 


As  suggested  by  the  flow  diagrams  in  Figure  6.17  and  Figure  6.21,  the  principal  data  product 
arising  from  the  processing  is  a  prioritized  dig  list  which  can  be  scored  against  ground-truth.  We 
have  tried  to  emphasize  throughout  this  section  that  our  procedures  and  ability  to  construct  a 
prioritized  dig  list  have  evolved  during  the  course  of  the  three  demonstrations  we  have  conducted 
and  have  reported  on  here.  Indeed,  the  SLO  demonstration  has  led  us  to  develop  and  document 
each  step  in  the  classification  process. 

While  the  prioritized  dig  list  is  the  primary  product  of  the  demonstrations,  there  are  also  many 
other  data  products  required  in  order  to  adequately  document  and  measure  the  performance  of 
the  MetalMapper  against  the  various  performance  objectives  that  we  have  enumerated  in  Section 
3. 
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7.  PERFORMANCE  ASSESSMENT 


In  this  section  we  will  assess  the  performance  of  the  MetalMapper  system  against  the 
performance  objectives  contained  in  Table  3.1.  So  as  to  avoid  needless  duplication,  we  will 
introduce  results  from  YPG  and/or  APG  where  appropriate.  Quite  apart  from  whether  we  met 
our  performance  goals  at  APG,  the  performance  measures  from  previous  demonstrations  provide 
useful  benchmarks  that  demonstrate  improvement  in  our  ability  to  efficiently  acquire  high- 
quality  data  effectively  interpret  them. 

7.1.  SITE  COVERAGE 

At  both  APG  and  SLO,  we  ran  our  dynamic  surveys  using  a  profile  offset  of  0.75m.  We  were 
able  to  meet  the  “tight”  0.75m  line  specification  for  the  SLO  detection  survey  by  processing  our 
profile  data  into  “split-cube”  profiles  wherein  each  MetalMapper  profile  becomes  7  split-cube 
profiles  with  13cm  offsets. 

For  convenience  in  processing  and  data  QC/QA,  we  divided  the  survey  area  into  13  sub-areas. 
Those  sub-areas  are  identified  in  Figure  6.4  and  again  in  Figure  7.1.  In  Figure  7.1  we  have 
zoomed  in  on  a  6-block  area  so  that  the  reader  can  see  some  detail  on  the  profiles.  To  be  sure, 
there  are  some  “gaps”  in  the  coverage.  However,  gaps  such  as  we  see  near  the  SW  comer  of 
block  M7  and  the  center  of  N7  reflect  the  presence  of  obstacles  (usually  outcropping  bedrock) 
that  were  impossible  for  the  survey  tractor  to  survey  over. 

7.1.1.  GPS  Data  Quality 

The  site  coverage  performance  objective  (section  3.1)  mentions  spurious  GPS  positions.  As  part 
of  our  field  QC,  we  always  plot  the  profile  tracks.  However,  the  quality  of  the  GPS  fix  is 
monitored  in  real  time  by  the  MetalMapper  acquisition  software  and  a  visual  alarm  is  shown  on 
the  acquisition  display  when  that  fix  is  no  longer  RTK  quality.  We  suspend  survey  operations 
when  the  quality  of  the  GPS  fix  is  not  RTK.  Based  on  these  2  QC  checks,  we  can  say  with  the 
utmost  confidence  that  our  GPS  positions  contain  virtually  no  spurious  points  or  drop-outs. 

Table  7.1:  Summary  of  footprint  coverage  for  the  13  sub-areas  comprising  the  11.8-acre  (4.78 
ha)  SLO  study  area.  The  instmment  footprint  was  assumed  to  be  0.75m. 


Sub  Area 

%  Coverage  Area  (m2) 

Area  Covered 

E 

99.26% 

6782.43 

6732.15 

NE 

98.34% 

5587.82 

5495.06 

NEa 

98.72% 

1611.35 

1593.92 

Nn 

99.96% 

3609.39 

3608.04 

Ns 

98.65% 

3642.05 

3592.73 

NWA 

99.79% 

1807.05 

1803.17 

NWB 

98.32% 

2719.86 

2674.14 

NWCa 

99.95% 

2593.71 

2592.50 

NWCb 

98.46% 

2807.85 

2764.71 

SA 

99.51% 

5398.66 

5372.26 

SWA 

100.00% 

2703.27 

2703.27 

SE-A 

98.26% 

4596.34 

4516.18 

SE 

99.86% 

3940.05 

3934.73 

Totals 

99.13% 

47799.83 

47382.86 
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7.1.2.  Area  Coverage  -  SLO 

To  quantify  the  percentage  of  area  covered  by  the  MetalMapper,  we  used 
UCEFOOTPRINTCOV  GX.  We  applied  the  GX  using  a  0.75m  footprint  to  each  of  the  13 
Geosoft  split  cube  GDB’s  using  a  polygonal  shape  file  that  coincides  with  the  sub-area  under 


705100  705200  705300  705400 


Figure  7.1:  An  index  map  showing  the  53  survey  blocks  of  the  SLO  study  area.  The  study 
area  was  divided  into  13  sub-areas.  The  brown  shading  is  actually  survey  points.  The  inset 
shows  the  actual  survey  profiles  for  a  small  area.  The  plotted  profiles  are  on  0.75m  centers 
and  represent  the  track  of  the  MetalMapper  survey  platform. 


investigation.  The  results  from  those  13  analyses  are  tabulated  in  Table  7.1.  More  than  99%  of 
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the  study  area  has  been  covered  with  no  gaps  greater  than  0.75m.  All  of  the  areas  that  were  not 
covered  were  inaccessible  to  our  survey  vehicle.  We  believe  that  these  results  show  that  the 
MetalMapper  has  substantially  met  the  objective  in  Table  3.1  for  survey  coverage. 


7.1.3. 


7.2.  Repeatability  of  Test  Strip  Calibrations 


7.2.1.  Dynamic  Calibrations 


Figure  6.7  is  a  pseudo-map  generated  by  compiling  60  repeated  profiles  over  the  test  strip  at 
SLO.  These  data  were  processed  in  exactly  the  same  way  as  we  processed  the  detection  maps 
for  the  study  area.  All  target  picks  shown  on  the  map  (yellow  triangle  symbols)  were  picked 
from  the  split  cube  grid  using  the  GRIDPICK  GX  from  Oasis  Montaj.  Because  the  test  strip  was 
not  entirely  clean  prior  to  the  seeding  of  the  10  targets  (T-001  through  T-010),  it  was  necessary 
to  delete  many  target  picks.  However,  none  of  the  remaining  picks  have  been  moved  or 
adjusted.  We  ended  up  with  a  set  of  59  points  for  each  target.  Each  target  pick  consists  of  an  X- 
Y  coordinate  point  (in  Local  coordinates)  plus  a  grid  value.  We  have  tabulated  the  means  and 


Table  7.2:  Summary  statistics  for  repeated  surveys  over  the  10  targets  seeded  in  the  test  strip 
at  SLO.  All  position  errors  are  given  in  meters.  The  peak  amplitude  values  are  in  units  of 
pT/s.  Note  that  the  local  Y  coordinate  is  along  the  survey  track.  The  columns  relevant  to 


repeatability  performance  objectives  are  hi 


flighted  in  blue. 
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standard  deviations  of  the  cross-track  (Local  X)  and  the  along  track  (Local  Y)  errors  between  the 
picked  anomaly  peak  and  the  ground-truth  for  each  target  as  provided  to  us  in  the  ESTCP 
demonstration  plan.  In  addition,  we  have  computed  the  mean  and  standard  deviations  for  the 
peak  amplitudes.  Recall  that  the  dynamic  survey  was  conducted  using  two  configurations  for  the 
deployment  of  the  MetalMapper  antenna  array:  a)  Skid-Mounted;  and  b)  Wheel-Mounted.  The 
height  of  the  antenna  array  above  ground  level  is  21cm  and  29cm,  respectively,  for  the  skid- 
mounted  and  wheel-mounted  deployments  so  it  is  necessary  to  report  repeatability  statistics  for 
the  peak  amplitudes  for  each  configuration.  In  Table  7.2,  we  have  summarized  the  statistics 
using  the  split  cube  map  (Figure  6.7)  together  with  an  automatic  peak  picker.  The  repeatability 
of  the  target  coordinates  is  very  good  with  standard  deviations  along  track  (Y)  of  0.1m  or  less. 
Over  some  targets  (e.g.,  T-002  the  error  has  a  bias  of  more  than  0.1m  (e.g.,  T-001  and  T-002). 
We  have  no  explanation  as  to  why  this  bias  (0.21m,  and  -0.35m).  What  is  significant  however  is 
that  the  standard  deviation  in  position  error  along  track  is  well  below  the  performance  objective 
of  0.25m  indicated  in  Table  3.1.  The  standard  deviations  of  peak  amplitudes  with  a  few 
exceptions  are  below  the  performance  standard  of  25%.  But  we  fail  to  meet  the  standard  of  15% 
or  less. 

We  tried  an  alternative  method  of  compiling  the  repeatability  statistics.  Instead  of  compiling  the 

Table  7.3:  Summary  statistics  for  repeated  surveys  over  the  10  targets  seeded  in  the  test  strip  at 
SLO.  All  position  errors  are  given  in  meters.  The  peak  amplitude  values  are  in  units  of  pT/s. 
These  statistics  are  based  on  line  by  line  picking  of  targets  and  more  extensive  editing  than 
those  shown  in  Table  7.2.  The  columns  relevant  to  repeatability  performance  objectives  are 
highlighted  in  blue. 
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test  strip  data  lines  into  a  map  (through  the  artifice  of  adding  lm  to  the  x-coordinates  of  each 
successive  lines,  we  compile  each  line  into  a  separate  map  consisting  of  7  (split-cube)  lines. 
Then  we  used  GRIDPEAK  GX  to  pick  the  lines,  followed  by  a  manual  edit  to  remove  spurious 
picks  and  (in  some  cases)  to  move  the  target  pick.  With  this  method,  it  was  much  easier  to  edit 
the  target  picks  to  make  sure  they  were  all  picking  the  same  target.  However,  the  process  of 
picking  had  to  be  repeated  59  times  and  was  time  consuming.  The  results  of  that  exercise  are  in 
Table  7.3.  Note  that  we  processed  these  data  in  UTM  coordinates.  So  in  order  to  report  along 
track  position  errors,  we  calculated  AY  based  on  the  azimuth  angle  32.4°  that  we  calculated  as 
the  azimuth  between  north  and  the  straight  line  extending  from  target  T-001  to  target  T-010  of 
the  test  strip.  The  results  are  similar  to  those  shown  in  Table  7.2.  The  MetalMapper  meets  the 
performance  goal  for  repeatability  of  the  anomaly  position.  However,  as  in  the  previous  results, 
the  repeatability  of  the  peak  anomaly  amplitudes  is  more  like  25%  (relative  to  the  associated 
mean  value)  as  opposed  to  the  performance  goal  of  15%. 


Discussion 

We  were  concerned  that  the  results  from  our  (dynamic) 
repeatability  tests  may  indicate  instability  that  we  do  not 
observe  when  we  compile  statistics  from  repeated  static 
measurements  wherein  the  platform  is  positioned  much 
more  precisely  over  the  target.  We  make  the  following 
observations: 

1.  When  viewed  in  a  local  coordinate  system  in 
which  the  line  x  =  0  represents  a  line  connecting 
target  T-001  to  target  T-010,  the  only  targets 
having  the  local  coordinate  x=0  are  the 
aforementioned  targets.  The  other  8  targets 
have  x-coordinates  with  non-zero  x-offsets  as 
large  as  0.28m  (see  Table  7.4). 

2.  The  test  strip  survey  was  conducted  as  a  single¬ 
profile  survey  in  which  the  vehicle  operator  navigated  along  a  pre-planned  straight-line 
profile  extending  from  T-001  to  T-010.  So  even  when  the  navigation  was  perfect  and 
the  antenna  platform  followed  the  line  x  =  0,  there  would  be  cross-track  offsets 
exceeding  10cm  between  the  antenna  reference  point  and  the  target  for  targets  T-002,  T- 
003,  T-005,  and  T-008.  Figure  7.2  is  a  base  map  showing  the  line  and  target  locations. 
The  target  symbols  are  approximately  0.8m  in  diameter.  The  figure  shows  that  the 
survey  lines  were  slightly  bowed  toward  the  grid  east  (local  x).  This  could  be  because 
our  navigation  software  does  not  account  for  the  GPS  antenna  height  and  platform 
attitude  in  real  time.  In  any  case,  the  figure  shows  the  profiles  to  be  biased  toward  grid 
east  in  the  center. 


Table  7.4:  Local  coordinates  for 
target  locations  in  the  test  strip  at 
SLO. 


Tgt 

X(m) 

Y(m) 

T-001 

0  . 

0  . 

T-002 

0.2845 

6.854 

T-003 

0.1112 

13.05 

T-004 

0.003835 

19.61 

T-005 

-0.1802 

26.08 

T-006 

0.06596 

32.57 

T-007 

-0.02991 

39.14 

T-008 

0 . 1058 

45.39 

T-009 

0.04832 

52.08 

T-010 

0  . 

58.67 
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3.  Analysis  shows  that  the 
ensemble  of  59  repeat  profiles 
can  be  contained  within  the 
limits  -0.4<x<0.4m.  Thus,  the 
variation  in  cross-track  distance 
from  the  target  may  be  40cm  or 
more. 

In  view  of  the  range  in  the  cross-track 
distance  from  a  target,  we  decided  to 
look  at  the  attenuation  of  the  peak 
amplitude  of  a  target  as  a  function  of 
profile  offset  from  the  target  center.  We 
varied  the  offset  in  5  cm  increments  out 
to  30cm.  We  analyzed  target  peaks  for 
both  split-cube  anomalies  and  for  the 
EM61-like  average  of  the  inner  5  Z- 
components  from  the  MetalMapper  cube 
array  (“Avg5IZ”). 

In  Figure  7.3  we  show  stacked  profiles 
for  the  for  the  7  MetalMapper  cubes  as 
they  move  across  the  target.23 
Regrettably,  the  MetalMapper  cubes  are 
ordered  according  to  their  cart  y-offset 
value  (along  track).  For  viewing  this  figure  it  would  be  more  convenient  if  they  were  ordered 
according  to  increasing  x-coordinate.  The  reader  can  refer  to  Figure  2.6  to  find  the  location  of 
the  receivers.  Cube  4  is  the  center  cube.  Cube  1  has  the  largest  cross-track  coordinate  (x=39cm) 
while  cube  7  has  the  smallest  cross-track  coordinate  (x  =  -39cm).  The  other  4  cubes  have  cross¬ 
track  coordinates  that  alternate  in  sign. 

At  YPG  and  APG,  we  used  a  detection  parameter  that  is  easier  to  compute,  the  Avg5IZ 
parameter.24  To  complete  this  analysis,  we  also  computed  the  Avg5IZ  profiles  as  a  function  of 
antenna  array  offset.  Those  profiles  are  shown  in  Figure  7.5.  Note  that  the  profiles  with  non¬ 
zero  offset  do  not  exhibit  perfect  symmetry  as  one  might  expect  from  a  real  single  loop  receiver. 
That  is  because  of  the  lack  of  bilateral  symmetry  in  the  MetalMapper  receiver  array. 

Both  Figure  7.3  and  Figure  7.5  show  that  the  attenuation  in  the  peak  anomaly  signal  is 
significantly  affected  by  offset  with  the  gradients  increasing  at  the  larger  offsets. 


SLO  Test  Strip 

Local  Coordinates 
Geometries,  Inc 


Figure  7.2:  Base  map  showing  the  SFO  Test 
Strip.  The  map  on  the  left  shows  the  strip  with  the 
locations  of  the  59  calibration  survey  lines  plotted 
in  brown.  The  red  numbered  symbols  indicate  the 
locations  of  the  10  seeded  targets.  For  reference, 
the  diameter  of  the  symbols  is  0.8m. 


23  The  reader  must  imagine  that  these  7  profiles  have  been  spread  out  laterally  on  13cm  centers 
and  shifted  along  the  y-axis  so  that  the  peaks  line  up. 

24  The  Avg5IZ  parameter  is  simply  the  average  z-component  from  cubes  2  through  6.  This 
parameter  behaves  much  like  a  large  single-loop  receiver  like  those  on  an  EM-61. 
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Figure  7.3:  Stacked  profiles  from  the  vertical  (z)  component  of  the  7  MetalMapper  receiver 
cubes  as  pass  over  a  buried  (horizontal)  60mm  target.  These  profiles  were  calculated  for 
different  profile  offsets  (x-coordinate)  from  the  target  center.  For  an  offset  of  x=0,  cube  0  has 
the  largest  peak  anomaly.  For  an  offset  of  30cm  (target  is  30cm  left  of  profile),  cube  7  has  the 
largest  peak  anomaly.  Cube  7  is  located  at  x=-39cm  relative  to  the  center  of  the  receiver 
array.  The  target  is  placed  at  depth  =  5 1cm,  which  is  the  nominal  depth  of  the  seeded  targets 
in  the  SLO  test  strip  plus  21cm  to  account  for  the  antenna  array  height  (skid-mounted). 
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Figure  7.5:  Parametric  profiles  of  the 
Avg5IZ  detection  scalar  for  the 

MetalMapper.  The  parameter  is  the 

cross-track  profile  offset  from  the  target 
location.  The  target  is  a  horizontal 

60mm  mortar  at  5 1cm  depth. 
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Rel  Peak  Amp  vs  Offset 


Profile  Offset  (m) 

Figure  7.4:  Variation  of  relative  peak 
amplitude  as  a  function  of  profile  offset  for 
split  cube  analysis  and  for  Avg5IZ.  The 
Avg5IZ  detection  parameter  is  similar  in 
behavior  to  that  of  an  EM61. 
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In  Figure  7.4  we  summarize  the  attenuation  of  target  amplitude  as  a  function  of  offset  for  the  two 
detection  types  (split-cube,  and  Avg5IZ).  Note  that  the  attenuation  for  both  cases  is  about  the 
same  for  offsets  out  to  about  15cm.  Then,  the  signal  strength  for  the  Avg5IZ  detector  falls  at  a 
significantly  faster  rate.  What  is  most  significant,  however,  is  the  relative  peak  signal  strength 
can  vary  by  as  much  as  20%  for  split  cube  analysis  and  by  35%  for  Avg5IZ.  This  study  shows 
that  great  care  must  be  taken  when  attempting  to  demonstrate  repeatability  in  dynamic  surveys. 
For  relatively  shallow  targets  such  as  those  in  the  SLO  test  strip,  the  survey  profile  must  be 
repeated  with  maximum  cross-track  errors  on  the  order  of  ±0.1 0-0. 15m.  We  simply  did  not  take 
sufficient  care  that  the  survey  followed  the  same  track. 

Because  we  processed  our  calibration  data  into  split-cube  lines,  we  were  able  to  make  rough 
estimates  of  both  the  along-track  and  cross-track  target  positions.  This  provides  us  with  data  that 
can  verify  whether  our  failure  to  achieve  the  ±15%  amplitude  repeatability  goal  is  at  least  in  part 
due  to  large  variations  in  offsets.  In  Figure 
7.6,  we  show  scatter  plots  of  the  (relative) 
peak  anomaly  value  as  a  function  of  the 
estimated  offset  distance  to  the  target.  These 
data  still  exhibit  more  scatter  than  we  would 
like.  Nonetheless,  they  suggest  a  relationship 
between  offset  and  the  peak  value.  In 
particular,  in  Figure  7.6B  the  quadratic  trend 
line  peaks  at  an  offset  of  approximately 
0.25m.  This  is  the  correct  offset  for  target  T- 
002  (see  Table  7.4). 

As  a  final  check  on  repeatability,  we  went 
back  to  our  APG  dynamic  calibration  data.  At 
APG,  we  laid  out  a  line  approximately  20m 
long  in  an  area  of  low  background.  At  the 
center  of  the  line,  we  buried  a  12-lb  shot  put  at 
a  depth  of  approximately  20cm.  We  ran  a 
dynamic  survey  consisting  of  2  reciprocal 
lines  over  this  target  a  minimum  of  twice  each 
day  during  those  days  that  we  were  surveying 
in  dynamic  mode.  Figure  7.7  is  a  base  map 
showing  the  calibration  lines  at  APG.  We 
have  plotted  the  location  of  25  lines  in  brown. 

The  red  circle  represents  the  mean  location  for 
the  anomaly  peak  from  the  calibration  sphere 
(shot  put).  The  inset  drawing  is  an  expanded 
view  of  a  1.5-m  section  of  the  line  that 
includes  the  target.  For  reference,  the  outside 
diameter  of  the  red  circle  is  0.75m.  The  cross-track  envelope  of  the  25  lines  that  have  been 
plotted  is  less  10cm.  The  statistics  for  the  peak  amplitude  of  our  calibration  sphere  at  APG  was 
37.90±3.83  (la)  for  a  repeatability  of  ±10%. 


Figure  7.6:  Scatter  plots  suggesting  that  part 
of  the  problem  with  poor  anomaly  amplitude 
repeatability  is  due  to  offset.  In  these  scatter 
plots,  the  horizontal  axis  is  the  cross-axis 
error  between  the  anomaly  peak  and  the 
target  position.  The  vertical  axis  is  the 
observed  anomaly  relative  to  its  mean. 
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Figure  7.7:  Base  map  upon  which  has  been  plotted  the  location  of  25 
dynamic  calibration  lines  at  APG.  The  red  circle  marks  the  target 
(shotput)  and  the  inset  shows  that  the  envelop  of  the  line  tranverse  to  the 
direction  of  motion  is  less  than  10cm. 


7.3.  Target  Detection 

At  SLO,  there  were  187  seeded  targets.25  The  MetalMapper  failed  to  detect  four  (4)  of  those 
seed  targets.  The  four  targets  we  missed  were  all  60mm  mortars.  Therefore,  we  achieved  the 


Figure  7.8:  Figure  showing  one  target  that  went  undetected  in  the 
MetalMapper  dynamic  survey.  Target  410  is  readily  discernible  on 
both  map  and  on  the  profiles.  Although  it  has  a  broad  areal  extent 
indicating  a  deep  target,  its  amplitude  fails  to  rise  above  the  detection 
threshold. 


25  We  obtained  an  unofficial  list  of  the  seeded  targets  from  Dean  Keiswetter  (SAIC). 
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stated  performance  goal  of  Pd  >=  98%. 

It  is  instructive  to  look  at  the  reasons  we  failed  to  detect  these  4  targets  (SLO  targets  59,  478  410, 
and  16.  All  4  of  the  targets  that  we  failed  to  detect  were  60mm  mortar  bodies.  Target  410  is 
typical  of  the  other  3.  The  issue  is  summarized  graphically  in  Figure  7.8.  The  anomaly  is 
obvious  in  both  the  map  and  the  profiles.  But  it  fails  to  rise  above  the  experimentally  verified 
detection  threshold.  The  “expert”  interpreter 
would  have  recognized  this  anomaly  as  likely 
arising  from  a  deep  target  because  of  its  broad 
extent  and  coherence  not  withstanding  its  low 
amplitude.  A  look  at  the  recovered  target 
object  (Figure  7.9)  explains  why  this  target 
had  a  lower  detection  signal  amplitude  than 
we  expected  for  a  60mm  object.  The  seeded 
target  was  a  60mm  mortar  (body  only)  target. 

We  defined  the  detection  threshold  based  on 
the  signal  amplitudes  for  a  60mm  M49A4 
(see  Figure  7.22),  the  type  of  60mm  mortar 
seeded  at  APG  and  YPG,  buried  in  the  test 
lanes  at  SLO,  and  provided  at  SLO  for  free- 
air  tests.  The  mortar  body  target  has  an 
electromagnetic  size  that  is  approximately 
half  the  size  of  the  complete  mortar  including 
fins  and  a  fuse.  Obviously,  the  detection  threshold  was  set  too  high.  However,  had  we  set  the 
threshold  to  1  uV/Am2  we  would  have  easily  doubled  or  perhaps  tripled  the  number  of  targets 
detected.  This  example  of  a  “false  negative”  in  detection  identifies  two  important  weaknesses 
in  basing  detection  on  the  criterion  that  the  peak  anomaly  must  exceed  a  threshold: 

1 .  The  threshold  value,  even  when  chosen  in  a  “principled”  manner  as  we  have  done,  does 
not  always  account  for  variations  in  the  configuration  of  the  “smallest”  target  of  interest. 
Nor  can  a  threshold  value  account  for  the  presence  of  a  target  of  interest  that  has  not 
been  identified  as  being  present  in  the  survey  area  and  therefore  was  not  considered 
when  establishing  setting  the  detection  threshold.  One  such  target,  a  37mm  projectile, 
was  found  during  the  target  recovery  at  SLO. 

2.  By  simply  lowering  the  threshold,  we  can  easily  double  or  triple  the  number  of  targets,  a 
very  high  percentage  of  which  are  bound  to  be  near-surface  clutter.  Most  low-amplitude 
anomalies  arise  from  small  bits  of  near-surface  clutter  having  very  different  anomaly 
characteristics  from  those  shown  in  Figure  7.8.  We  believe  that  by  using  the  fact  that 
each  sample  of  MetalMapper  dynamic  data  actually  includes  vector  measurements  at  7 
different  spatial  data  points  relative  to  the  platform  position  can  be  used  to  recognize  the 
difference  between  anomalies  having  roughly  the  same  peak  amplitude  arising  from 
small  objects  at  shallow  depths  and  large  objects  at  deeper  depths.  Such  a  capability  can 
provide  the  basis  for  a  better  “physics-based”  method  of  target  detection  or  anomaly 
screening. 
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7.4.  Analysis  and  Classification  Performance 

In  section  6  we  tried  to  show  how  our  capabilities  for  making  “principled”  decisions  have 
developed  throughout  the  course  of  this  project.  The  SLO  project  served  as  a  catalyst  that 
permitted  us  to  develop  a  methodology  that,  in  most  respects,  achieves  the  performance 
objectives  set  out  in  the  ESTCP  demonstration  plan.  The  bases  for  discussion  in  this  section  are 
the  scoring  results  that  we  received  from  IDA  in  early  November  2009.  In  Figure  7.10  below, 
we  have  reproduced  and  annotated  the  IDA  ROC  curves.  These  ROC  curves  summarize  the 
scoring  results  representing  a  partial  dig  (1064  items26  from  a  list  of  1408)  of  the  MetalMapper 
target  list. 


7.4.1.  Maximize  Correct  Classifications  -  Munitions  (see  Section  3.4) 
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Figure  7.10:  IDA  ROC  curves  for  the  two  discrimination  stage  target  lists  submitted  in 
connection  with  the  SLO  classification  study.  The  differences  in  the  lists  a  the  result  of  using 
different  data  sets  for  training  the  ANN  and  different  versions  of  the  ANN  software.  In  the 
insets,  we  have  identified  the  4  false  negative  targets  that  occur  beyond  the  MM  threshold. 
Note  that  the  same  4  targets  are  involved  in  both  target  lists  (albeit,  in  different  order). 


The  ROC  curves  (Figure  7.10)  speak  directly  to  the  objectives  for  maximizing  the  correct 
classification  for  both  TOI  and  clutter.  With  regard  to  TOI,  the  results  show  that  at  our  operating 
threshold,  in  both  cases  we  had  4  false  negatives,  two  of  which  were  very  “hard”  in  the  sense  that 
our  classifier  indicated  that  it  was  highly  probable  that  these  objects  were  clutter.  However, 
target  list  2  (Figure  7.1 0B)  distinguishes  itself  from  its  companion  in  that  significantly  fewer 
unnecessary  digs  are  required  before  the  last  TOI  is  eventually  found. 


We  have  examined  the  dig  results  from  the  four  targets  associated  with  the  hard  negatives.  In  the 
expanded  ROC  curves  (inset  Figure  7.10  A  &  B)  the  4  targets  are  identified  according  to  their 
MetalMapper  ID  and  Master  List  IDs,  respectively,  with  the  “/”  character  as  a  separator.  The 


26  The  test  data  used  by  IDA  consists  of  1064  targets  of  which  206  were  designated  TOI  and  the 
balance  (858)  as  clutter  or  non-ordnance  items. 
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37mm  target  is  the  only  target  that  is  an  obvious  “miss”  that  could  have  (should  have)  been 
avoided  by  a  comprehensive  visual  examination  of  the  polarizability  curves.  After  assembled 
our  prioritized  dig  list,  we  did  in  fact  perform  a  partial  visual  QC  involving  perhaps  100  targets 
on  either  side  of  our  operating  threshold.  We  did  this  in  order  to  identify  a  set  of  targets  bounded 
on  one  side  by  the  operating  threshold  (i.e.,  the  boundary  between  “High  Confidence  Not 
Munitions”  in  Figure  6.21)  and  on  the  other  side  by  targets  designated  as  “High-Confidence- 
Munitions”  targets.  These  “Ambiguous”  targets  are  those  that  we  believe  are  associated  with 
analyzable  data  for  which  we  get  ambiguous  classifications.  However,  in  both  target  lists  the 
37mm  target  (the  only  one  in  the  test  data  set)  was  very  strongly  identified  as  clutter  (non¬ 
munition).  The  problem  here  is  that  our  classifier  was  not  trained  to  identify  a  target  of  the  size 
and  shape  of  the  37mm.  In  our  retrospective  study,  we  added  the  polarizability  curves  for  a 
37mm  from  our  library  of  munitions  types  for  APG  (see  Figure  7.11).  Our  library  matching 
algorithm  identified  the  37mm  with  very  high  probability.  Therefore,  we  conclude  that  we  will 
be  able  to  detect  and  classify  37mm  using  library  matching  provided  we  add  a  37mm  type  to  our 
matching  library  (a  6th  type)  and  that  we  include  specimens  of  the  9-component  feature  vector 
extracted  from  static  measurements  of  37mm  projectiles. 
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Figure  7.11:  Principle  polarizability  curves  for  a  37mm  projectile.  The  curves  on  the  left 
(A)  represent  the  average  6  or  8  curve-sets  extracted  from  data  sets  observed  over  37mm 
projectiles  in  the  Calibration  Lanes  at  APG.  The  curves  on  the  right  were  extracted  from 
static  data  set  SLOStatA00292.tem,  the  only  data  set  observed  over  this  37mm  target.  This 
target  was  miss-classified  as  Clutter. 


The  second  hard  false  negative  that  we  wanted  to  examine  is  the  60mm  mortar  found  in 
association  with  MM-ID  1177  (Master  List  ID  775).  The  polarizability  curves  together  with  the 
dig  photo  are  in  Figure  7.13.  A  segment  of  the  detection  map  together  with  the  associated  data 
profile  is  shown  in  Figure  7.12.  For  reference  on  the  map,  the  target  symbols  have  a  diameter  of 
lm.  The  profiles  show  the  average  5IZ  detector  (top  -  red)  and  the  7  vertical  components  plotted 
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together  in  the  bottom  panel  as  color- 
coded  profiles.  The  horizontal  coordinate 
is  the  easting  value.  But  the  profile  is 
generally  in  the  NW-SE  direction  so  the 
horizontal  scale  as  indicated  by  the  axis 
grid  is  misleading.  The  peak  1176 
adjacent  to  and  SE  of  1177  is  on  the 
MetalMapper  target  list  but  had  not  been 
excavated  at  the  time  that  IDA  released 
the  scoring  to  SLO  demonstrators.  In  any 
case,  the  polarizability  curves  suggest  a 
shallow  equi-dimensional  object  which  is 
consistent  with  the  60mm  tail  boom 
segments  sitting  on  the  board.  The 
anomaly  amplitude  is  less  10  pT/s  which 
means  it  is  only  slightly  above  the 
detection  threshold  for  a  deep  mortar.  Yet 
the  solution  for  the  target  position 
indicates  a  target  at  a  depth  of  13  cm.27  It 
appears  therefore  that  the  anomaly 
produced  by  the  tail  boom(s),  which  no 


Figure  7.12:  Detection  profdes  and  a  segment  of  the  associated  split  cube  detection 
map  for  MetalMapper  Cued  ID  target  number  1177.  The  anomalies  seem  to  reflect  a 
shallow  target  such  as  the  tail  booms  found  (presumably)  above  the  deeper  60mm 
mortar. 
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Figure  7.13:  MM  Cued  ID  No.  1177  and  the 
corresponding  dig  photo.  The  target  in  the  hole 
is  a  60mm  mortar.  The  polarizability  curves 
appear  to  represent  the  associated  tail  booms 
and/or  fuse. 


27  The  z  values  given  in  the  polarizability  plot  is  with  respect  to  the  platform  reference  point. 
That  point  is  21cm  above  the  ground  surface. 
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doubt  were  found  above  the  deep  mortar  shown  in  the  photo,  totally  overpowers  the  low-signal 
level  from  the  mortar.  This  is  a  case  where  the  interference  from  several  shallow  sources 
obscures  the  deeper  TOI.  The  best  and  only  chance  we  have  of  mitigating  this  type  of  anomaly 
is  to  recognize  that  it  is  a  complex  response  that  cannot  be  analyzed  and  therefore  must  be  dug. 

The  remaining  two  false 
negative  targets  have  MM- 
ID’s  1718  and  1782. 

These  targets  correspond 
to  2.36-in  rocket  parts. 

The  dig  photos  and  the 
associated  polarizability 
plots  are  shown  in  Figure 
7.14.  On  target  List  1 
(Figure  7.10A),  neither  of 
these  targets  is  very  “hard” 
false  negative  and  require 
only  a  few  extra  10’s  of 
digs  before  they  are  found. 

However,  on  List  2,  1718 
(at  about  300  extra  digs, 

Figure  7.1  OB)  is  very 
close  to  the  37mm  target 
(292).  Therefore,  the  fact 
that  we  can,  in  a 
“principled”  way 

reorganize  our  dig  list  to 
remove  the  37mm  doesn’t  help  us  much  because  we  have  target  1718  with  about  the  same  dig 
priority.  In  both  cases,  the  polarizability  curves  are  in  no  way  suggestive  that  the  target  is  a  TOI. 


SLOStatAOl 718-Principal  Polarizability  Curves  (Type  1) 


SLOStatAOl  782-Principal  Polarizability  Curves  (Type  1) 


Figure  7.14:  Polarizability  curves  and  dig  results  for  MM-ID 
Numbers  1718  and  1782,  two  of  4  false  negatives  in  both  target 
lists  submitted  to  IDA. 


MM-ID  1718:  The  profdes  for  the  dynamic  survey  line  crossing  directly  over  this  target 
together  with  a  segment  of  the  split  cube  detection  map  that  includes  the  target  are  shown  in 
Figure  7.15.  Although  there  is  some  noise  in  the  individual  cube  profiles,  both  the  map  and  the 
profiles  suggest  a  single  target.  Target  1717  is  located  less  than  lm  to  the  SE  of  1718  and  looks 
to  be  the  2nd  peak  of  the  anomaly.  Our  inversion  software  predicts  that  the  source  point  for  the 
target  lies  0.4m  east  of  the  target  pick  (center  of  target  circle).  The  polarizability  curves  for 
MetalMapper  targets  1717  and  1718  and  the  size  estimates  (P0x=428  in  Figure  7.18  right) 
indicate  that  the  parameters  most  likely  belong  to  the  60mm  tail  boom  shown  in  the  dig  photo 
rather  than  the  rocket  motor.  The  polarizability  response  of  an  isolated  rocket  motor  is 
significantly  larger  and  shows  good  symmetry  and  large  aspect.  This  target  intrigued  us  partly 
because  it  is  a  hard  false  negative,  but  also  because  the  dig  reports  that  the  target  was  found 
laying  on  the  surface.  How  then  could  we  have  missed  a  target  like  that?  In  looking  through  our 
field  notes,  we  came  across  a  note  for  file  SLOStatA01715,  a  static  data  point  acquired  in  the 
immediate  vicinity  of  1718.  The  note  (see  Figure  7.16A)  indicates  that  a  measurement  on  an 
unpicked  point  was  acquired  near  or  over  a  “rocket  shaft”  that  had  obviously  been  moved 
recently  because  it  was  laying  on  top  of  the  grass.  It  is  likely  that  the  rocket  motor  shown  in 
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Figure  7.15:  Detection  profiles  and  map  segment  showing  the  target  MM-ID  No  1718, 
classified  incorrectly  as  clutter  (false  negative).  The  spikes  and  offsets  in  the  individual  cube 
profiles  are  the  result  of  a  problem  we  had  with  shorting  turns  in  one  of  our  transmitter  loops. 
The  problem  was  corrected  in  the  early  days  of  the  dynamic  survey.  For  reference,  the  yellow 
target  symbols  have  a  diameter  of  about  lm.  The  static  data  files  SLOStatA01715  and 
SLOStatA02204  were  acquired  at  positions  close  to  1718.  Field  notes  (for  1715)  indicate  the 
presence  of  a  “rocket  shaft”  on  the  surface  that  obviously  been  moved  from  its  original 
position.  The  notes  also  indicated  that  1715  DID  NOT  correspond  with  a  target  pick. 

Figure  7.18  was  found  in  the  immediate  vicinity  (i.e.,  within  a  radius  of  lm  or  so)  and  lumped  in 
with  the  60mm  tail  boom.  The  ground-truth  indicates  the  target  was  found  within  a  distance  of 
0.3m  of  the  location  of  MM  1718. 
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A)  Excerpt:  Field  notes  (p.  SL061  6/13/09) 


SLOStatAOl  715-Principal  Polarizability  Curves  (Type  1) 
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SLOStatA02204-Principal  Polarizability  Curves  (Type  1) 


B) 


C) 


Figure  7.16:  Field  note  describing  target  near  1715  as  a  “rocket  shaft”  (A)  together  with  the 
polarizability  curves  for  1715  and  a  repeat  (2204).  The  polarizability  curves  both  look  like 
rocket  parts.  These  two  data  sets  are  within  a  meter  or  two  of  MM-ID  1718. 
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Figure  7.18:  Polarizability  curves  for  MetalMapper  static  files  1717  and  1718 
corresponding  to  SLO  target  1475.  The  curves  are  not  indicative  of  a  target  like 
the  2.36-in  rocket  motor  shown  on  the  white  board.  More  likely,  they  characterize 
the  60mm  tail  boom  that  is  shown. 


SLOStatAOl  71 8-Principal  Polarizability  Curves  (Type  1) 


SLOStatAOl  71 7-Principal  Polarizability  Curves  (Type  1) 


SLOStatA00954-Principal  Polarizability  Curves  (Type  1) 
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SLOStatAOl  782-Principal  Polarizability  Curves  (Type  1) 


Figure  7.17:  Polarizability  curves  for  two  2.36-in  “empty”  warheads.  The  target  on  the  left 
is  from  the  training  dig,  the  one  on  the  left  is  from  the  test  dig.  Although  target  1782/1541 
was  deeper  than  954/1260,  both  targets  exhibit  similar  behavior  in  the  early  time. 
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MM-ID  1782:  Relatively  speaking,  target  number  MM-ID  1782  (Master  ID  1541)  is  the  least 
important  of  the  4  false  negative  targets  that  are  labeled  on  both  panels  of  Figure  7.10.  The 
position  of  this  target  is  roughly  the  same  on  both  ROC  curves  (Figure  7.10  A  &  B)  and  requires 
only  a  few  unnecessary  digs  past  the  operating  point  (dark  blue  dot)  to  get  to  the  target.  We 
mention  it  here  only  for  completeness.  We  would  like  to  note  however,  that  the  target  (Master 
ID  1541)  is  similar  in  both  appearance  and  in  the  behavior  of  its  principal  polarizability  curves  to 
target  SLO  Master  ID  1322  that  was  belatedly  labeled  “MD”  after  a  more  careful  examination. 
The  polarizability  curves  and  photos  of  MetalMapper  targets  954  and  1782  provide  a  comparison 
between  the  two  targets,  one  labeled  “MD”  and  the  other  labeled  “TOI”.  The  similar  behavior  of 
the  polarizability  curves  suggests  that  target  1782  is  cracked  or  otherwise  physically  damaged  in 
a  way  similar  to  that  of  954. 

Remarks:  The  forgoing  discussion  contains  reasonable  explanations  3  of  the  4  false  negative 
targets  appearing  in  both  target  lists.  After  considerable  analysis,  we  can  find  no  explanation  for 
why  our  polarizability  parameters  do  not  indicate  a  60mm  mortar  other  than  the  mortar  was 
reasonably  deep  (26cm)  and  were  two  60mm  tail  booms  found  above  the  mortar.  The 
polarizability  curves  seem  consistent  in  shape  and  size  with  the  tail  booms  and  therefore  it  is 
most  likely  that  the  low-level  signal  from  the  mortar  was  overwhelmed  by  much  larger  signals 
arising  from  shallow  tail  boom  targets. 

7.5.  Maximize  Correct  Classification  of  Clutter 

In  the  test  data  set,  there  were  a  total  of  857  clutter  objects.  The  results  from  List  2  (  Figure 
6.28B  ),  shows  that  after  ~300  unnecessary  digs,  we  have  excavated  all  the  TOI  (Pd  =  100%). 
Therefore  we  found  all  the  TOI  after  a  total  of  506  digs  and  would  be  able  to  safely  leave  the 
remaining  557  clutter  targets  in  the  ground.  This  represents  a  reduction  of  65%  in  false  alarms 
and  thus  we  exceed  the  performance  goal  of  a  30%  reduction.  We  note,  however,  that  if  we  use 
our  List  1,  it  would  take  approximately  906  digs  before  we  had  excavated  the  last  TOI.  This 
represents  a  reduction  of  only  18%. 
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Discussion:  Prior  to  submitting  our  second  target  list  (List  2),  we  made  what  appeared  to  be  a 
minor  correction  in  the  rules  algorithm  that  we  apply  to  the  stream  of  the  ANN  decision  scores 
and  the  target  matching  scores.  These  changes  did  not  appear  to  affect  our  training  ROC. 
However,  we  applied  the  same  (revised)  decision  rules  to  both  Lists  1  and  2  and  plotted  new 
ROC  curves.  These  curves  are  shown  in  Figure  7.19.  In  making  these  ROC  curves,  we  used  all 


Figure  7.19:  ROC  curves  for  SLO  test  results.  These  curves  were  generated  from  ground- 
truth  provided  by  IDA.  However,  we  used  the  ranks  we  generated  for  the  15  category  4 
targets  rather  than  score  them  as  “unnecessary  digs”.  The  ROC  curve  for  List  1  (A)  also 
reflects  a  minor  change  to  the  decision  rules  that  was  made  after  the  original  List  1  was 
submitted.  This  change  was  applied  to  List  2  before  submission. 

category  4  targets  since  the  ground-truth  provided  by  IDA  showed  that  our  decisions  on  those 
(15)  targets  were  all  correct.  Moreover,  the  List  1  ROC  reflects  a  minor  change  to  the  rule-based 
decision  algorithm  that  we  made  after  we  submitted  List  1  and  before  we  submitted  List  2. 
These  curves  reflect  exactly  the  same  rules.  Note  that  the  rule  change  moved  the  very  hard  false 
negative  in  Figure  7.19A  down  to  about  Pfp=40%.  This  means  that  now,  only  342  unnecessary 
targets  are  dug  before  we  get  to  Pd=100%.  With  this  minor  change  to  our  rules,  List  1  also 
achieves  the  performance  goal  of  a  30%  reduction  in  unnecessary  digs.  There  is  no  change  to  the 
performance  of  List  2  (about  300  unnecessary  digs  at  Pd=100%). 

7.6.  Operating  Point  Specification  (No-Dig  Threshold) 

The  operating  points  for  Lists  1  and  2  are  indicated  as  dark  blue  points  in  Figure  7.10.  In  both 
cases,  this  operating  point  is  at  Pd=98%  (202/206).  At  that  point,  there  have  been  approximately 
90  unnecessary  digs.  This  leaves  a  total  of  767  clutter  objects  (85%)  correctly  classified  and  thus 
exceeds  the  performance  goal  of  >30%  of  the  clutter  objects  correctly  classified  at  the 
demonstrator-specified  operation  point. 
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7.7.  Minimization  of  Cannot  Analyze  Points 

Of  the  1063  targets  that  were  scored,  there  were  15  category  4  points  (“Cannot  Analyze”). 
Those  points  were  points  for  which  the  fit  statistic  was  less  than  75.  The  points  represent  1.4% 
of  all  the  targets  analyzed.  Using  the  ground-truth  from  the  test  dig,  we  determined  that  the 
discrimination  results  for  all  15  category  data  points  were  correct.  Although  we  do  not  feel 
comfortable  with  completely  eliminating  a  screening  rule  that  relegates  data  points  with  poor  fit 
to  category  4,  in  retrospect  we  think  that  we  were  too  conservative  in  assigning  the  Fit<75% 
threshold.  By  assigning  a  slightly  lower  threshold  (e.g.,  Fit  <  70%)  we  can  eliminate  half  of  the 
category  4  points  with  little  or  no  danger  of  introducing  a  false  negative.  In  any  case, 
MetalMapper  has  clearly  met  the  performance  goal  of  >90%  of  all  targets  being  analyzable. 

7.8.  Correct  Estimation  of  Parameters 

In  the  MetalMapper  data  set,  there  were  a  total  of  187  data  files  whose  positions  correlated  with 
the  known  position  of  a  seeded  target.28  We  used  the  inversion  results  from  these  data  files  as  the 
basis  forjudging  how  well  we  did  on  our  parameter  estimates.  To  attain  the  performance  goals, 
it  is  first  necessary  to  correctly  estimate  the  3 -dimensional  position  of  the  target.  Provided  we 
have  a  good  estimate  of  position,  estimates  of  the  principal  polarizability  curves  are  straight 
forward.  Errors  in  the  magnitude  of  the  principal  polarizability  curves  are  proportional  to  the 
position  error  according  to  a  power-law  [25]  and,  hence,  a  bad  estimate  of  the  former  guarantees 
a  worse  estimate  of  the  latter. 

7.8.1.  Position  Estimates 

We  computed  the  3-D  position  errors  between  the  position  of  the  seeded  target  (as  measured 
after  the  dig)  and  the  position  estimated  from  the  associated  MetalMapper  static  data  point.  We 
analyzed  the  horizontal  positions  and  the  depths  separately. 

Depth  Errors:  The  distribution  of  depth  errors  for  the  187  seed  targets  analyzed  is  shown  in  the 
histogram  (Figure  7.20).  The  mean  and  the  standard  deviation  (4.9cm  and  6.5cm,  respectively) 
are  also  noted  on  the  diagram.  The  solid  red 
curve  is  a  Gaussian  curve  with  the  indicated 
mean  and  standard  deviation  that  has  been 
normalized  to  the  number  scale  of  the 
histogram.  These  data  show  that  depth  errors 
are  reasonably  approximated  by  a  normally 
distributed  random  process.  The  mean  value 
of  4.9cm  shows  that  the  MetalMapper  depth 
estimates  are  biased  too  deep.  We  have  not 
been  able  to  explain  this  bias  in  the  depth 
estimate,  thought  the  bias  was  not  seen  in 
other  surveys  at  APG  and  Camp  Butner.  In 
any  case,  the  figure  shows  that  the 
MetalMapper  depth  estimates  easily  meet  the 


Depth  Errors:  SLO  Seed  Targets 


Depth  Error  (cm) 

Figure  7.20:  Depth  error  histogram  for  187 
seed  targets  at  SLO. 


28  There  were  an  additional  34  MetalMapper  data  files  that  were  associated  with  seeded  targets 
but  were  not  as  close  to  the  target  as  the  primary  target  file. 
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performance  goal  of  ±10  cm  (la).  The  systematic 
error  is  attributable  to  how  the  depths  were 
measured. 

Horizontal  Position  Estimates:  In  Figure  7.21 
we  have  plotted  the  horizontal  error  as  a  polar 
plot.  The  plot  has  been  annotated  with  the  mean 
and  standard  deviation  for  both  the  error  radius 
(Ar)  and  the  error  azimuth  (4>).  The  plot  shows 
that  the  errors  are  biased  by  4.5cm.  It  is  possible 
that  this  bias  reflects  a  misalignment  between  the 
GPS  antenna  (elevated  1 .48m  above  the  platform 
reference  point)  and  the  normal  to  the  Z 
transmitter  coil  that  passes  through  the  center 
cube.  It  is  also  possible  (and  perhaps  more  likely) 
that  the  bias  reflects  an  output  bias  in  the  X-Y 
accelerometers  on  the  attitude  sensor.  With  the 
GPS  antenna  waving  around  at  a  height  of  1.48m 
above  the  platform,  an  error  of  only  1.8°  would 
introduce  a  bias  of  4.5cm.  However,  even  with  the  bias  included  in  the  error,  the  horizontal 
position  estimates  are  well  within  the  performance  goal  of  15cm  (la). 

7.8.2.  “Beta”  Estimates 

The  term  “Beta”  as  used  in  Table  3.1  refers  to  the  principal  polarizability  transients  and/or 
composite  values  derived  there  from  by  numerically  integrating  the  principal  polarizability 
transients  over  a  specific  time  gate.  As  we  have  stated  previously,  these  principal  polarizability 
transients  are  the  source  of  all  the  intrinsic  information  about  the  target.  In  the  October, 
NRL/SAIC  undertook  to  make  static  free-air  measurements  of  each  of  the  seeded  target  items 
that  were  used  during  the  SLO  field  demonstration  using  the  TEMTADs  system.  Typically,  each 
target  was  measured  in  three  attitudes  (flat,  nose  up,  and  nose  down)  at  a  single  depth.  These 
data  were  reduced  using  the  SAIC  dipole-based  inversion  program  to  yield  a  set  of  3  principal 
polarizability  curves  for  each  target  and  each  attitude.  Unfortunately,  the  ground-truth  that  was 
recorded  failed  to  identify  a  specific  seed  item  with  a  specific  target  site  or  SLO  Master  List 
target  number.  However,  from  the  ground-truth  information  provided,  we  were  able  to  identify 
186  MetalMapper  targets  as  “Seed”  targets.  Moreover,  the  ground-truth  that  we  were  provided 
allowed  us  to  associate  each  of  those  targets  with  6  different  UXO  types.29  Therefore, 
statistically  at  least,  we  have  a  basis  for  measuring  how  well  we  were  able  to  estimate  the  target 
“Betas”  by  comparing  “Beta”  values  extracted  from  the  free-air  measurements  that  SAIC 
conducted  on  SLO  seed  items,  and  similar  “Beta”  values  extracted  from  MM  data  sets 
corresponding  to  the  same  type  of  seeded  target. 

SAIC  was  kind  enough  to  supply  us  with  a  well  documented  set  of  principal  transient  curves 
resulting  from  the  inversion  of  TEMTADs  data  sets  acquired  over  the  seeded  items.  As  we 
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Figure  7.21:  Polar  scatter  plot 

showing  the  errors  in  the  estimation  of 
horizontal  position  for  187  seed  targets 
at  SLO. 


29  Although  there  were  only  4  targets  of  primary  interest  (60mm/81mm/4.2-in  mortars  plus  2.36- 
in  rockets),  we  learned  60mm  and  81mm  mortar  seed  items  consisted  of  “mortar  bodies”  and 
“complete  mortars”  (i.e.,  with  fuse  and  fins). 
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indicated  above,  this  documentation  identifies  the  specific  target  type  (e.g.,  “60mm  mortar”, 
“60mm  mortar  body”,  “4.2-in  mortar”,  etc)  but  was  not  able  to  identify  a  particular  target 
specimen  with  a  particular  SLO  Master  List  target  number.  All  TEMTADs  data  were  acquired 
using  a  base  acquisition  frequency  of  10  Hz  that  resulted  in  principal  polarizability  curves  with  a 
time  bandwidth  of  25ms.  Moreover,  SAIC  reduction  procedures  apply  a  delay  time  correction  of 
-28  ps  and  an  amplitude  correction  of  0.175  to  their  data  before  inversion.  The  delay  correction 
accounts  for  analog  delay  in  the  acquisition  system  and  was  empirically  determined  by  SAIC  so 
that  test  data  acquired  with  the  TEMTADs  system  better  matches  the  theoretical  curves  for  a 
sphere.  Likewise,  the  amplitude  factor  was  established  for  similar  reasons  before  SAIC  was 
provided  with  information  about  the  gain  settings  in  the  TEMTADs  data  acquisition  system.  In 
its  reduction,  MetalMapper  does  not  apply  these  correction  factors30  and  so  we  have  “backed 
out”  the  SAIC  corrections  from  their  data  in  order  that  we  can  fairly  compare  “Betas”  extracted 
from  either  system. 


30  There  is  validity  in  the  time  delay  correction.  However,  we  have  not  attempted  to 
experimentally  establish  its  value.  We  are  confident  that  the  voltage  amplitudes  that  we  measure 
are  correct  with  a  precision  of  approximately  1%. 
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Comparison  of  “Beta”  Curves  -  For  each  target  class  (e.g.,  60mmM49A4)  we  have  a  number 
of  free-air  measurements  acquired  with  the  MTADs  system  and,  likewise,  a  number  of 
measurements  made  over  in  situ  targets  with  the  MetalMapper.  In  Figure  7.22,  we  have  plotted 
the  principal  polarizability  transients  of  all  the  measurements  for  both  60mmM49A4  and  81mm 
targets.  Although  there  is  some  scatter  in  the  individual  curves  due  to  different  target  attitude 
and  perhaps  minor  differences  between  various  instances  of  the  target  types,  the  resulting 
geometric  mean  principal  curves  (heavy  red/green/blue  curves)  are  virtually  identical.  Similar 
results  were  obtained  for  4  other  target  types:  a)  60mm  M49A4  Body;  b)  81mm  Body;  c)  2.36-in 
rocket;  d)  4.2  in  mortars.  It  is  difficult  to  compare  the  geometric  mean  curves  since  they  are 
based  on  different  time  gates  and  time  bandwidths.31  In  principle,  it  would  be  possible  to 
quantitatively  compare  the  resulting  geometric  mean  transient  curves  using  the  same  curve 
matching  procedure  that  we  applied  for  discrimination.  However,  because  of  the  different  time 


Figure  7.22:  Figure  comparing  individual  (fine  lines)  and  geometric  mean  principal 
polarizability  curves  (heavy  red/green/blue  curves)  corresponding  to  seed  targets  acquired  on 
the  one  hand  in  free  air  with  the  NRL/SAIC  MTADs  system  (top  row)  and  on  the  other  hand 
acquired  in  situ  at  SLO  with  Geometries’  MetalMapper.  The  geometric  mean  curves  derived 
from  the  data  acquired  with  the  two  systems  are  virtually  identical  when  viewed 
logarithmically. 


31  The  TEMTADs  data  were  acquired  with  a  time  bandwidth  of  25ms  and  a  smaller  decimation 
factor  (5%). 
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bandwidths  and  logarithmic  window  sampling  parameters,  we  elected  a  simpler  procedure  to 
affect  a  quantitative  comparison  as  we  explain  below. 

Comparison  of  Composite  “Beta”  or  P0’s  -  In  section  6.3.2  we  explained  how  we  derived 
certain  “meta”  parameters  from  the  principle  polarizability  curves.  The  simplest  of  these  meta¬ 
parameters  are  the  3  Po’s,  which  are  nothing  more  than  a  numerical  integration  of  the  principal 
polarizability  curve  over  a  specified  time  window.  This  integration  is  standard  in  our  inversion 
program  MMRMP  and  it  was  easily  performed  on  the  SAIC  principle  polarizability  curves  from 
the  SLO  seed  target  objects.  We  used  the  composite  time  gate  100<t<8333  jus  to  compute  the 
parameters  POx,  POy,  and  POz.  From  these  three  parameters  we  calculated  the  transverse 
polarizability  parameter  Pot  (also  explained  in  section  6.3.2).  We  can  visually  compare  how  the 
6  target  types  plot  on  a  scatter  plot  as  we  shown  in  Figure  7.23.  We  have  calculated  the 
geometric  means  and  the  geometric  standard  deviation  for  each  of  the  6  target  groups.  These 
values  are  presented  as  Table  7.5.  The  table  shows  that  we  are  able  to  reproduce  the  actual  beta 
values  (assumed  to  be  those  measured  in  free-air  by  SAIC)  with  an  error  (1  a)  of  considerably 
less  than  ±20,  which  is  our  performance  objective. 


TEMTADs:  Frcc-Aii  "yS's"  MetalMapper  (Static)  "fi's" 


100  1000  10000  100000  100  1000  10000  100000 

p,x(™3) 

A)  TEMTADs:  Free-air  SLO  Seed  Targets  B)  Metal  Mapper:  In  situ  SLO  seed  targets 

Figure  7.23:  Scatter  plots  comparing  composite  “Betas”  or  Po’s  as  acquired  from  SLO  seed 
target  objects  with  the  TEMTADs  system  in  free-air  and  the  MetalMapper  system  with  in  situ 
measurements  of  the  targets  as  buried  at  SLO. 
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Table  7.5:  Table  comparing  TEMTADs  “Betas”  to  “Betas”  recovered  by  MetalMapper  for  seed 
targets  at  SLO.  The  table  shows  that  we  are  able  to  recover  primary  “Betas”  (POx)  with  errors 
less  than  ±15%  which  is  well  under  our  performance  objective. _ 


Type 

TEMTADs 

MM 

Error 

TEMTADs 

MM 

Error 

Geom.  Mean  -  POx  (cmA3) 

Geom.  Mean  -  POt  (cmA3) 

tgt236in 

5167 

4790 

7.30% 

538 

568 

5.60% 

tgt42in 

23566 

20853 

11.51% 

7804 

7304 

6.40% 

tgt60mm 

2524 

2403 

4.79% 

638 

571 

10.47% 

tgt60mmBody 

1374 

1325 

3.52% 

430 

426 

0.91% 

tgt81mm 

7560 

7018 

7.17% 

1896 

1842 

2.84% 

tgt81mmBody 

7468 

6460 

13.49% 

2295 

1747 

23.86% 

7.9.  Dynamic-Mode  Survey  Productivity 

Our  performance  objective  for  dynamic-mode  survey  productivity  was  1.2  Acres/day.  In  Table 
7.6,  we  have  tabulated  the  work  time  for  each  of  the  sub-areas  that  we  defined  within  the  overall 
demonstration  area.  These  time  intervals  (At)  represent  the  time  between  the  first  and  last 
dynamic  calibration  of  the  day  (or  last  calibration  for  a  particular  sub-area  survey)  and  therefore 
include  time  for  breaks/breakdowns.  What  those  time  intervals  do  not  include  is  the  time  for  set¬ 
up  and  take-down  each  day.  We  estimate  that  time  to  be  0.5  hr/day.  We  based  our  production 
estimates  on  the  number  of  work  hours.  At  SLO,  we  generally  worked  about  9-10  hrs  in  the 
field.  Therefore,  we  converted  hourly  production  to  daily  production  by  multiplying  by  9  hr/day. 
Based  on  the  table,  we  met  our  slightly  exceeded  our  production  goal  (0.054  ha/hr  =  0.49  ha/day 
=  1.22  acre/day). 

We  also  tabulated  dynamic  data  production  for  the  MetalMapper  demonstration  at  APG  (see 
Table  7.7).  At  APG,  the  mapping  mode  survey  productivity  was  significantly  larger  (-0.07  ha/hr 
-0.56  ha/day  =1.4  acre/day).32  This  was  due  in  large  part  we  think  because  the  survey  lines  over 
both  Indirect  Fire  and  Direct  Fire  areas  were  approximately  100m.  This  reduced  the  number  of 
turn-arounds  at  the  end  of  each  survey  line. 

The  tables  show  that,  at  both  SLO  and  APG,  we  have  met  or  exceeded  our  performance  goal  for 
dynamic-mode  survey  productivity  (1.2  acre/day). 


32  For  APG,  we  based  our  daily  production  estimate  on  an  8-hr  field  day  (plus  0.5hr  for  set-up 
and  take-down).  Based  on  the  9-hr  field  day  that  we  used  for  the  SLO  estimates,  the  comparable 
production  rate  would  be  0.63  ha/day. 
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Table  7.6:  Table  showing  dynamic  survey  productivity  at  SLO 


Sub-Area 

Date 

Work  Time 

Production 

File  No 

Blks 

Area 

Start 

End 

At 

At  (hr) 

Rte  (ha/hr) 

Rte  (ha/d) 

1st 

Last 

(ha) 

5/25/09 

12:33 

17:45 

5:12 

5.20 

0.035 

0.18 

1 

47 

2 

0.18 

5/26/09 

7:12 

14:51 

7:39 

7.65 

0.035 

0.27 

48 

101 

3 

0.27 

5/26/09 

15:48 

17:48 

2:00 

2.00 

0.046 

0.43 

102 

116 

6 

0.54 

5/27/09 

7:16 

16:14 

8:58 

8.97 

117 

232 

NWC 

5/28/09 

6:59 

7:46 

0:47 

0.78 

233 

254 

SWA 

5/28/09 

9:13 

15:41 

6:28 

6.47 

0.042 

0.27 

255 

338 

3 

0.27 

SA 

5/29/09 

6:59 

16:51 

9:52 

9.87 

0.055 

0.54 

339 

424 

6 

0.54 

SE 

5/30/09 

7:11 

12:14 

5:03 

5.05 

0.088 

0.54 

425 

516 

9.5 

0.855 

SE-A 

5/30/09 

12:52 

17:32 

4:40 

4.67 

517 

576 

E 

5/31/09 

6:01 

15:59 

9:58 

9.97 

0.068 

0.68 

577 

687 

7.5 

0.675 

NE 

6/1/09 

7:24 

17:13 

9:49 

9.82 

0.057 

0.56 

688 

785 

6.2 

0.558 

NE-A 

6/2/09 

7:41 

12:22 

4:41 

4.68 

0.035 

0.16 

786 

861 

1.8 

0.162 

N 

6/9/09 

5:42 

15:47 

10:05 

10.08 

0.071 

0.72 

862 

1061 

8 

0.72 

NWB-Rep 

6/15/09 

8:45 

11:23 

2:38 

2.63 

0.103 

0.27 

1062 

1105 

3 

0.27 

1.1.  Totals 

87.83 

0.054 

0.46 

53 

4.77 

Table  7.7:  Dynamic  survey  productivity  at  APG. 


Area 

Date 

Work  Time 

Production 

File  No 

Area 

Start 

End 

At 

At  (hr) 

Rte  (ha/hr) 

Rte  (ha/d) 

1st 

Last 

(ha) 

CG 

9/15/08 

10:09 

13:11 

3:02 

3.03 

0.040 

0.32 

1 

52 

0.12 

BG 

9/16/08 

9:09 

14:48 

5:39 

5.65 

0.028 

0.23 

56 

119 

0.16 

IDF 

9/19/08 

7:17 

13:46 

6:29 

6.48 

0.084 

0.67 

140 

211 

1.3 

IDF 

9/20/08 

6:26 

11:29 

5:03 

5.05 

212 

271 

IDF 

9/22/08 

6:58 

10:59 

4:01 

4.02 

272 

323 

DF 

9/26/08 

11:10 

14:15 

3:05 

3.08 

0.084 

0.67 

1 

43 

0.76 

DF 

9/27/08 

6:25 

12:21 

5:56 

5.93 

44 

114 

Totals 

33.25 

0.070 

0.56 

2.34 
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7.10.  Static-Mode  Calibration  Repeatability 

We  placed  a  steel  calibration  sphere  in  the  test  pit  at 
SLO  after  we  completed  the  static  measurements  on 
specimen  targets  of  interest.  During  the  time  period  Jun 
7-16,  2009,  we  returned  to  measure  that  calibration 
sphere  20  times.  Each  measurement  consisted  of 
reacquiring  the  target  using  the  acquisition  software  to 
guide  the  operator  backed  to  the  known  target  position 
(measured  and  stored  when  we  buried  the  target).  Each 
static  data  set  was  inverted  with  MMRMP  to  get 
estimates  for  the  3D  target  position,  and  size.  We  have 
tabulated  the  statistics  in  Table  7.8.  And  in  Figure  7.24 
we  have  plotted  the  estimates  of  (horizontal)  platform 
and  target  position.  The  table  and  figure  show  that  we 
are  able  to  consistently  move  the  MetalMapper  platform 
to  within  a  few  centimeters  of  a  designated  target 
position.  The  repeatability  of  target  position  estimates  is  at  the  sub-centimeter  level.  And 
finally,  the  estimate  of  target  size  (RMS(Po))  is  within  ±4.0%  (la),  and  the  measure  of  signal 
strength  (SNR)  is  repeatable  to  ±4.4%  (la).  These  results  show  that  we  have  easily  met  our 
MetalMapper  performance  goal  (±10%  (la))  for  repeatability  based  on  measuring  a  reference 
target. 


Cal  Ball  Position  Estimates  (18) 
105o90°  75° 


240°  ^ — — -^300° 
255  °270  °285  ° 


Figure  7.24:  Polar  plot  of 

horizontal  platform  and  target 
position  estimates  for  the  SLO 
static  calibration  data  points. 


Table  7.8:  Summary  statistics  for  static  repeatability  at  SLO.  The  top  table 
section  speaks  to  the  ability  to  repetitively  re-  position  the  platform  over  the  same 
point.  The  bottom  section  contains  the  statistics  for  the  estimates  of  target  position 
and  size. 


Platform  Position 

<P 

[°) 

Ar  (cm) 

z  (cm) 

Mean 

a 

Mean 

a 

Mean 

a 

10.9 

±112 

3.3 

±2.1 

N/A 

±0.8 

Target  Posil 

tion  &  EM  Size 

SNR 

<P 

[°) 

Ar  (cm) 

z  (cm) 

RMS(P0  )(cm3) 

Mean 

a 

Mean 

a 

Mean 

a 

Mean 

a 

Mean 

a 

-17.7 

±103 

0.7 

±0.5 

40.4 

±0.6 

4078 

±164 

301.02 

13.37 

7.11.  Static-Mode  Survey  Productivity 

Our  performance  goal  at  SLO  was  to  acquire  greater  than  200  static  data  points  per  day.  As 
shown  in  Table  7.9  and  Table  7.10.  These  tables  show  that,  at  SLO,  we  improved  static 
productivity  by  approximately  39  pts/hr  versus  29  pts/hr  at  APG.  At  SLO,  we  measured  494  pts 


ESTCP  Final  Report  MM-0603: 


101 


MetalMapper 


on  our  best  day.33  The  improved  production  rates  reflect  the  front-mounted  antenna  deployment. 
Secondly,  we  decreased  the  stacking  time  by  a  factor  or  2  because  of  the  generally  better  SNR  at 
SLO. 


Table  7.9:  Static-mode  survey  productivity  at  Camp  SLO. 


Area 

Date 

Work  Time  Production 

File  Index 

Start 

End 

At 

At  (hr) 

Rte  (pts/hr) 

Rte  (pts/d) 

1st 

Last 

Tst  Strip 

6/3/09 

7:27 

8:40 

1:13 

1.22 

17 

201 

2 

22 

Tst  Pit 

6/3/09 

9:03 

10:18 

1:15 

1.25 

27 

23 

56 

Tst  Strip 

6/8/09 

7:39 

8:35 

0:56 

0.93 

13 

242 

57 

68 

NWC 

6/8/09 

9:22 

14:25 

5:03 

5.05 

43 

1 

217 

NWC 

6/10/09 

14:37 

17:40 

3:03 

3.05 

41 

197 

364 

489 

NWB 

6/10/09 

8:23 

11:48 

3:25 

3.42 

43 

218 

363 

NWA 

6/10/09 

17:41 

18:34 

0:53 

0.88 

40 

490 

524 

NEPIus 

6/11/09 

8:35 

15:21 

6:46 

6.77 

40 

405 

525 

795 

E 

6/11/09 

15:33 

18:18 

2:45 

2.75 

57 

796 

952 

E 

6/12/09 

8:13 

13:10 

4:57 

4.95 

51 

494 

953 

1205 

SEPIus 

6/12/09 

14:18 

18:08 

3:50 

3.83 

60 

1206 

1434 

Splus 

6/13/09 

8:30 

17:01 

8:31 

8.52 

50 

449 

1435 

1859 

N 

6/14/09 

9:46 

17:25 

7:39 

7.65 

42 

375 

1860 

2178 

Stat-Rep 

6/15/09 

12:40 

18:42 

6:02 

6.03 

17 

155 

2179 

2282 

Stat-Rep 

6/16/09 

8:47 

17:33 

8:46 

8.77 

24 

216 

2283 

2492 

Totals 

65.07 

39 

354 

2559 

33  At  SLO,  our  access  to  the  site  was  unrestricted.  We  generally  stayed  a  full  10  hrs  at  the  site. 
Daily  production  rates  at  SLO  partly  reflect  longer  work  days  than  at  APG.  For  comparing 
production  rates  it  is  better  to  use  the  hourly  rates. 
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Table  7.10:  Static-mode  survey  productivity  at  APG 


Area 

Date 

Work  Time 

Production 

File  Index 

Start 

End 

At  (hr) 

Rte  (pts/hr) 

Rte  (pts/d) 

1st 

Last 

CG(5%) 

9/16/08 

7:05 

9:13 

2.13 

38 

300 

4 

83 

CG(10%) 

9/16/08 

13:49 

14:43 

0.90 

34 

305 

2 

32 

CG(10%) 

9/16/08 

6:40 

7:33 

0.88 

42 

33 

69 

BG 

9/17/08 

7:43 

14:47 

7.07 

27 

247 

71 

264 

BG 

9/18/08 

6:19 

13:27 

7.13 

35 

266 

512 

BG 

9/19/08 

6:30 

7:02 

0.53 

26 

513 

526 

IDF 

9/22/08 

14:16 

16:31 

2.25 

25 

213 

2 

58 

IDF 

9/23/08 

6:35 

14:50 

8.25 

25 

60 

263 

IDF 

9/24/08 

6:45 

14:44 

7.98 

28 

265 

486 

IDF 

9/25/08 

6:04 

14:03 

7.98 

27 

488 

703 

IDF 

9/26/08 

6:24 

9:59 

3.58 

28 

705 

806 

DF 

9/30/08 

14:25 

14:37 

0.20 

25 

225 

2 

6 

DF 

10/1/08 

6:23 

13:10 

6.78 

22 

7 

158 

DF 

10/2/08 

8:15 

16:45 

8.50 

30 

160 

413 

DF 

10/4/08 

6:28 

10:47 

4.32 

34 

416 

561 

Totals 

68.50 

29 

229 

1961 
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8.  COST  ASSESSMENT 


This  section  should  provide  sufficient  cost  information  such  that  a  professional  involved  in  the 
field  could  reasonably  estimate  costs  for  implementation  at  a  given  site.  In  addition,  this  section 
should  provide  a  discussion  of  the  cost  benefit  of  the  technology.  The  following  subsections  with 
detailed  discussions  and  examples  should  be  provided. 

8.1.  COST  MODEL 

In  Table  8.1  we  provide  a  cost  model  for  the  deployment  of  the  MetalMapper  technology. 


Table  8.1  Cost  Model  for  a  MetalMapper  Deployment 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

Instrument  cost 

Component  costs  and  integration  costs 

•  Engineering  estimates  based  on  current 

development 

•  Lifetime  estimate 

Track  consumables  and  repairs 

•  $100K-$120K 

Mobilization  and 
demobilization 

Cost  to  mobilize  to  site 

Derived  from  demonstration  costs 

•  $20K/$15K 

Instrument  setup 
costs 

Unit:  $  cost  to  set  up  and  calibrate 

Data  Requirements: 

•  0.5  hr 

•  2 

•  2/day 

•  $500/day 

Survey  costs 
(Dynamic) 

Unit:  $  cost  per  hectare 

Data  requirements: 

•  Hours  per  hectare  -  1 8  hr 

•  Personnel  required  -  2 

•  $7.2K/ha 

Survey  costs 
(Static) 

Unit:  $  cost  per  300  Targets 

Data  requirements: 

•  Hours-  9  hr 

•  Personnel  required  -  2 

•  $3.6K 

Detection  data 

processing  costs 

Unit:  $  cost  per  hectare 

Data  requirements: 

•  Time  required  -  18  hr 

•  Personnel  required  -  1 

•  $6.6K 

Discrimination  data 
processing  costs 

Unit:  $  per  300  anomaly  anomalies 
•  Time  required  -  9hr 

Personnel  required  -  lUnit: 

•  $3.3K 
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8.1.1.  Equipment  Costs 

Major  sub-systems  comprising  the  MetalMapper  system  together  with  their  respective  cost 
estimates  are  listed  in  Table  8.2. 


Table  8.2:  Major  components  comprising  a  MetalMapper  system 


MetalMapper  Component  Name 

Cost 

Antenna  platform  and  Data  Acquisition  System  (DAQ) 

$65K 

RTK  GPS  System 

$30K 

Platform  attitude  sensor 

$5K 

Vehicle  Deployment 

$5K-$20K 

Software 

$10K-$20K 

Antenna  Platform  and  DAQ  -  The  antenna  platform  and  the  DAQ  comprise  the  proprietary 
hardware.  The  platform  can  be  deployed  “as  is”  as  a  human-powered  cart.  The  instrument 
containing  the  DAQ  can  be  mounted  on  a  pack-frame  and  carried  by  an  operator.  In  most  cases, 
the  system  is  integrated  with  an  RTK  GPS  system  that  periodically  generates  a  NMEA  position 
string  to  be  read  by  the  DAQ.  And  for  precision  static  measurements,  an  orientation  module  is 
mounted  on  the  antenna  platform  to  sense  the  three  attitude  angles  (heading,  pitch,  and  roll)  of 
the  platform. 

RTK  GPS  System-The  location  of  the  antenna  platform  is  normally  measured  using  an  RTK 
GPS  system  in  which  the  roving  antenna  is  fixed  to  the  antenna  platform.  The  cost  of  an  RTK 
GPS  system  (base  +  rover)  is  estimated  in  Table  8.2.  However,  the  cost  will  vary  depending  on 
the  manufacturer  and  the  features  of  the  system. 

Platform  Attitude  Sensor-We  use  an  attitude  sensing  module  that  contains  a  2-axis 
accelerometer  to  measure  pitch  and  roll  angles  and  a  3 -axis  fluxgate  magnetometer  that  provides 
reference  heading  to  magnetic  north.  The  module  output  is  in  the  form  of  ASCI  coded  RS232C 
text  strings. 

Deployment  -  The  MetalMapper  can  be  deployed  either  as  a  man-powered  cart  or  as  a  vehicle- 
towed  system.  Deployment  in  the  vehicle-towed  mode  requires  a  tow-vehicle  such  as  small  lawn 
tractor.  There  is  a  large  range  of  possibilities  for  tow  vehicles  and  therefore  we  have  provided  a 
cost  range  for  the  deployment. 

Software-The  cost  of  software  required  to  process  MetalMapper  data  depends  on  the  data  type. 
Map  data  is  processed  using  Geonics  Oasis  Montaj  (OM)  software.  We  supply  a  set  Geosoft 
eXecutables  (GX’s)  that  permit  MetalMapper  data  files  to  be  loaded  into  a  Geosoft  database 
(GDB).  Standard  features  of  OM  are  used  to  edit  and  further  process  the  data,  detect  targets,  and 
manage  the  resulting  target  lists.  OM  is  not  required  to  process  static  data.  Target  parameters 
are  extracted  using  Geometries’  proprietary  modeling  software  based  on  the  point  dipole  model. 
Results  of  this  target  parameterization  are  placed  in  a  Microsoft  Access  database.  Further 
processing  for  discrimination  can  be  performed  using  standard  commercially  available  software 
such  as  MS-Excel,  MS-Access,  and  others. 

8.1.2.  Mobilization  and  Demobilization 

The  cost  of  mobilization  includes  the  cost  of  equipment  preparation  and  checkout  plus  the  cost  of 
transportation  to  the  job  site.  The  estimated  costs  are  based  on  our  experience  in  mobilizing 
from  Grand  Junction,  CO  to  job  sites  at  YPG  (~1000mi),  and  APG  (~2000mi).  The  estimated 
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cost  includes  labor,  transportation,  and  per  diem.  Demobilization  does  not  include  cost  for 
equipment  preparation. 

8.1.3.  Instrument  Setup  and  Calibration 

Instrument  setup  and  calibration  procedures  require  approximately  Vi  hr  at  the  beginning  and  at 
the  end  of  each  field  day.  These  procedures  include  setting  up  the  GPS  system,  conducting 
operation  checks,  and  performing  static  and/or  dynamic  calibration  surveys. 

8.1.4.  Dynamic  Survey  Costs 

The  cost  estimate  is  based  on  a  crew  cost  of  $3600/day.  The  estimated  areal  production  is  based 
on  survey  profiles  on  0.75m  offsets.  Survey  speeds  are  less  than  0.5m/s.  Production  rates  can  be 
improved  by  increasing  the  survey  speed.  Production  of  up  to  1  ha/day  is  possible  at  a  survey 
speed  of  lm/s. 

8.1.5.  Static  Survey  Costs 

The  cost  of  static  surveys  is  based  on  an  average  production  rate  of  300  static  points/day.  At 
APG,  we  were  able  to  acquire  up  to  200  pts/day.  At  SLO  we  averaged  well  over  300  pts/day 
with  the  antenna  array  mounted  to  the  front  loader  of  a  Kabota  4WD  tractor.  Production  rates 
will  vary  depending  on  the  type  of  deployment. 

8.1.6.  Detection  Data  Processing 

A  single  data  analyst  can  process  the  data  produced  by  a  MetalMapper  crew.  It  requires 
approximately  lhr  of  the  analyst’s  time  per  hour  of  field  data  production.  Processing  includes 
loading  the  data,  producing  a  detection  map,  and  picking  targets.  It  also  includes  sequencing  the 
target  lists  for  static  surveys. 

8.1.7.  Discrimination  Data  Processing 

As  with  detection  surveys,  a  competent  data  analyst  is  able  to  keep  up  with  the  processing  of  the 
data  acquired  by  the  MetalMapper  field  crew  (~300  static  points/day).  Costs  are  figured 
accordingly.  Note  that  these  costs  do  not  include  the  costs  of  training  the  discrimination 
algorithm. 

8.2.  COST  DRIVERS 

The  cost  estimates  in  Table  8.1  use  higher  labor  costs  that  reflecting  the  personnel  involved  in 
early  demonstrations.  These  labor  costs  will  no  doubt  drop  by  as  much  as  1/3  as  field  operations 
and  data  processing  tasks  become  routine  and  are  accomplished  by  lower-level  personnel.  The 
productivity  of  detection  surveys  is  likely  to  improve  by  as  much  as  a  factor  of  2  as  we  increase 
survey  speed.  And  finally,  the  productivity  of  the  data  analyst  stands  to  improve  markedly  as 
processing  tasks  are  integrated  and  automated.  It  is  likely  that  cost  of  processing  will  decrease 
by  as  much  as  50%. 

8.3.  COST  BENEFIT 

The  cost  benefit  of  employing  of  advanced  EMI  technology  such  the  MetelMapper  can  only  be 
realized  to  the  extent  that  regulators  gain  sufficient  confidence  in  the  technology  that  they  are 
comfortable  letting  targets  that  have  been  classified  “High  Confidence  Not  Munitions”  remain 
undug  in  the  ground.  Based  on  the  cost  estimates  provided  in  Table  8.1,  the  unit  cost  for  “Cued 
ID”  analysis  of  a  target  with  the  MetalMapper  is  approximately  $20/target  with  the  costs  likely 
to  drop  rapidly  down  to  $  10/target  .  The  unit  cost  to  dig  has  been  estimated  to  run  between 
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$100-$200/target.  The  scoring  results  for  the  MetalMapper  showed  that  fully  98%  of  the 
munitions  would  have  been  dug  with  a  corresponding  false  positive  dig  rate  of  less  than  10%  if 
the  targets  had  been  dug  according  to  the  prioritized  dig  list.  This  suggests  that  by  employing 
such  dig  list  in  a  very  conservative  manner  at  least  half  of  the  targets  could  be  left  in  the  ground 
thereby  reducing  the  digging  costs  by  half.  Since  the  MetalMapper  static  survey  and  data 
analysis  costs  are  approximately  10-20%  of  the  cost  to  dig,  the  overall  cost  of  remediation  would 
be  reduced  by  30-40%  (i.e.,  dig  50%  of  targets  and  conduct  a  cued  ID  target  survey  with  the 
MetalMapper  of  100%  of  the  targets  at  a  cost  of  10-20%  of  the  cost  to  dig). 

9.  IMPLEMENTATION  ISSUES 

In  Section  8  above,  we  highlighted  the  most  important  issue  affecting  the  implementation  of  the 
MetalMapper  technology.  Clearly,  the  need  is  to  prove  to  the  authorities  that  regulate  and 
supervise  the  remediation  of  sites  contaminated  with  UXO  that  the  MetalMapper  technology  can 
significantly  reduce  the  cost  remediation.  We  think  that  the  ESTCP  program  office  together  with 
regulators  and  demonstrators  are  making  some  headway  with  their  live  site  demonstration 
program.  Thus  far,  successful  demonstrations  at  Camp  Siebert,  AL  and  Camp  San  Luis  Obispo 
have  been  completed.  The  ESTCP  program  office  plans  to  continue  the  live  site  demonstration 
program.  In  2010,  a  be  a  third  live  site  demonstration  is  scheduled  to  be  conducted  at  Camp 
Butner  located  in  the  Raleigh- Durham  area  of  North  Carolina. 

Even  with  the  reduction  in  the  unit  survey  costs,  the  cost  for  deploying  the  MetalMapper  will 
remain  significantly  higher  than  the  cost  for  commercially  available  EMI  technology  such  as  the 
EM61.  And  clearly,  this  higher  cost  is  justified  provided  that  regulating  authorities  accept  the 
discrimination  capability  of  the  system  and  use  the  results  to  reduce  the  number  of  unnecessary 
digs.  UXO  contractors  will  no  doubt  begin  to  deploy  advanced  EMI  systems  such  as  the 
MetalMapper  only  when  regulating  authorities  start  specifying  the  use  of  such  technology  in  the 
big  requests  relating  to  UXO  site  remediation.  Clearly,  it  is  important  that  the  MetalMapper 
participated  in  the  demonstration  at  SLO  and  performed  well.  It  is  equally  important  that  several 
other  data  processing  demonstrators,  using  MetalMapper  data,  performed  equally  as  well. 

The  MetalMapper  system  is  available  as  a  commercial  product  by  Geometries,  Inc  (San  Jose, 
CA).  It  can  be  deployed  in  the  field  using  personnel  with  same  technical  skills  as  those  who 
routinely  conduct  UXO-related  Geophysical  Mapping  surveys.  In  some  respects,  data 
processing  is  more  demanding.  However,  we  are  confident  that  this  too  can  be  routinely 
performed  by  geophysical  data  processors  with  a  few  days  training. 
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Appendix  A:  Points  of  Contact 


POINT  OF 
CONTACT 

Name 

ORGANIZATION 

Name 

Address 

Phone 

Fax 

E-mail 

Role  in  Project 

Mark  Prouty 

Geometries,  Inc 
2190  Fortune  Drive 
San  Jose,  CA 
95131 

408-428-4212 

408-954-0902 

markp@geometrics.com 

Principal  Investigator 

ESTCP  Final  Report  MM-0603. 


110 


MetalMapper 


